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CHAPTER  I 


INTRODUCTION 


To  every  man  there  comes  in  his  lifetime  that  special  moment 
when  he  is  figuratively  tapped  on  the  shoulder  and  offered  that 
chance  to  do  a  very  special  thing,  unique  to  him  and  his  talents. 
What  a  tragedy  if  that  moment  finds  him  unprepared  or  unqualified 
for  that  work. 

Winston  Churchill 


Background 

Inherent  in  every  leadership  position  is  the  responsibility  to 
efficiently  and  effectively  manage  scarce  resources.  The  ability  to 
carry  out  this  responsibility  requires  a  development  of  the  necessary 
management  skiiis.  ifanagement  development  is  perhaps  even  more 
critical  to  the  Air  Force,  partly  because  of  the  magnitude  and  unique¬ 
ness  of  its  operations,  and  partly  because  of  the  manner  in  which  the 
Air  Force  obtains  its  higher  level  managers.  Unlike  the  civilian 
sector,  which  is  frequently  able  to  recruit  personnel  to  fill  higher 
level  management  positions,  the  Air  force  must  develop  its  own.  Since 
the  Air  Force  has  accepted  the  fact  that  there  are  very  few  "born 
managers"  (AFTfi  25-1,  1964;36),  management  development  must  be  an  Air 
Force  program. 

At  the  outset  it  is  important  to  discuss  the  terminology  u^ed 
in  this  thesis.  The  terms  "career  development”,  "officer  development" 
and  "officer  career  development”  are  used  interchangeably  in  the  liter 
ature.  As  used  in  this  thesis  they  are  all  taken  to  mean  the  same 
thing.  Officer  development  encompasses  bo*h  the  technical  and 
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management  development  training  that  every  officer  needs  to  progress  in 
a  given  career  field.  The  focus  of  this  thesis  is  on  the  management 
development  aspects  of  officer  development.  It  is  assumed  that  while 
technical  training  requirements  vary,  all  Air  Force  officers  have 
similar  management  development  needs. 

Lt.  General  Rogers  (1975:4)  viewed  officer  development  as  the 
integration  of  functional  technical  training,  advanced  academic  training, 
professional  military  education  and  experience  throughout  the  individ¬ 
ual's  military  career.  Through  this  process,  critical  attitudes,  knowl¬ 
edge  and  skills  are  increased  to  meet  the  challenge  of  more  responsible 
positions. 

Guidelines  for  officer  development  are  provided  in  two  Air 
Force  publications,  AFiri  25-1,  U.S.  Air  Force  management  Process,  and 
AFR  36-23,  Officer  Career  Development. 

"The  military  management  concept  is  that  an  accumulation  of 
diverse  experience  is  essential  to  develop  an  officer's  capacity  to 
work  effectively  at  command  level  and  above"  (AFffi  25-1,  1964:36). 
According  to  AFR  36-23  (1979:4-1)  "the  primary  purpose  of  career 
management  is  to  insure  that  qualified  officers  are  available  to  take 
on  responsibility  within  the  defense  establishment,"  Two  specific 
objectives  are  to  (1)  develop  officer  qualifications  to  meet  Air  Force 
needs  and  (2)  orovide  the  training  and  rotation  of  assignments  needed  to 
develop  their  capabilities. 

Traditionally,  management  development,  has  been  defined  as  "an 
informal  process  consisting  of  the  person  handling  a  variety  of 
assignments  along  a  prescribed  route  of  advancement"  (Boehm  and  Hoyle, 
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1977:204).  It  can  be  both  training  designed  to  impart  knowledge 
necessary  for  effective  job  performance,  and  a  planned  succession  of 
job  assignments  to  gain  knowledge  and  experience  (Boehm  and  Hoyle,  1977) 

Craig  (1967:406)  defines  management  development  as  "the  planned 
experience,  guided  growth,  and  training  opportunities  provided  for 
those  who  perform  the  management  functions."  It  has  two  specific 
objectives,  one  immediate  and  one  long  range.  The  immediate  objective 
is  to  increase  the  present  performance  of  individuals.  The  long  range 
abjective  is  to  prepare  those  individuals  with  recognized  potential  for 
future  advancement  and  responsibility  (Craig,  1967).  This  long  range 
objective  is  vitally  important  to  the  Air  Force  for  at  lease  two  reasons 
(1)  continual  reductions  in  both  numerical  and  experience  levels  of 
officer  personnel,  and  (2)  a  promotion  system  which  dictates  that  all 
future  Generals  will  come  from  within  the  lower  ranks  of  the  officer 
force. 

Historically,  management  development  is  not  a  new  concept,  but 
its  importance  has  become  more  obvious  as  the  technology  and  complexity 
of  today's  organizations  increase.  Prior  to  World  War  II,  "the  implied 
assumption  of  management  development  generally  was  that  talent  woula 
automatically  reveal  itself  through  the  application  of  skill  and 
industry  at  work"  (English  and  fflarebane,  1977:88).  This  approach  proved 
to  be  effective  in  meeting  the  demands  for  management  talent  resulting 
from  accelerated  organizational  growth.  Firms  realized  that  formally 
organized  and  planned  development  programs  were  essential  to  maintain 
an  adequate  resource  of  trained  managers  (English  and  Marchone,  1977:68) 
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The  Air  Force  has  also  come  to  recognize  the  need  for  planned 


development  of  the  officer  force. 


In  the  past,  officers  improved  themselves  and  their  sub¬ 
ordinates  without  the  aid  of  a  formal  program.  The  Air  Force 
sought  to  develop  most  of  its  officers  within  the  organizational 
structure.  It  was  reasoned  that,  by  exposure  to  an  operational 
environment  and  on-the-;cb  training,  officers  would  develop  the 
professional  and  technical  skills  required  to  insure  their  moti¬ 
vation,  effectiveness,  and  performance.  However,  the  Air  Force 
today,  because  of  the  increasing  technical  complexity  and  sophis¬ 
ticated  management  needs,  requires  a  formal  career  development 
program  (AFR  36-23,  1979s 1-1). 


Management  development  in  the  Air  Force  is  provided  through 
both  formal  management  development  programs  and  informal  cn-the-job 
training.  The  formal  programs  include,  but  are  not  limited  to. 
Professional  Military  Education,  tne  Air  Force  Institute  of  Technology, 
and  specialized  management  development  courses.  Informal  management 
development  is  based  primarily  on  coaching,  counseling  and  job  rotation. 

A  very  important  aspect  of  any  management  development  program 
is  the  feedback  provided  to  the  individual  on  present  performance.  This 
feedback  can  be  used  to  identify  strengths  and  weaknesses  in  management 
skills,  and  serve  as  a  basis  for  initiating  an  individual  program  to 
correct  deficiencies. 


Statement  of  the  Problem 

Although  there  are  both  formal  and  informal  programs  for  officer 
management  development,  weaknesses  exist  in  both.  Formal  management 
development  opportunities  such  as  Professional  Military  Education  and 
the  Air  Force  Institute  of  Technology  programs  are  available  to  only  a 
portion  of  the  officer  force. 
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Both  personal  experience  and  the  results  of  the  Air  Force 
Quality  of  Life  Survey,  Third  Edition  (McNichois,  I960),  indicate  that 
a  weakness  also  exists  in  the  informal  methods  of  management  develop¬ 
ment.  According  to  the  survey,  61.8  percent  of  the  officers  surveyed 
agreed  that  more  supervision  of  member  performance  and  behavior  is 
needed  at  lower  levels  within  the  Air  Force.  Additionally,  38. A  percent 
indicated  they  seldom  or  never  received  feedback  from  their  supervisor 
about  their  job  performance.  It  appears  that  a  better  way  must  be 
found  for  providing  feedback  to  officers  to  assist  in  their  develop¬ 
ment  as  Air  Force  managers. 

Approach  to  the  Problem 

The  assessment  center  is  a  technique  which  may  be  useful  for 
management  development  of  Air  Force  officers.  The  assessment  center 
has  been  used  extensively  in  business  and  government  since  1956  for  a 
number  of  purposes,  tut  primarily  for  selection  of  individuals  for 
management  jobs.  The  Task  Force  on  Development  of  Assessment  Center 
Standards  (Moses,  1975* 2— 3 )  defined  assessment  center  as  follows: 

in  summary,  an  assessment  center  consists  of  a  standard¬ 
ized  evaluation  of  behavior  based  on  multiple  inputs.  Multiple 
trained  observers  and  techniques  are  used.  Judgments  about 
behavior  are  made,  in  part,  from  specially  developed  assessment 
simulations.  These  judgments  are  pooled  by  the  assessors  at  an 
evaluation  meeting  during  which  all  relevant  assessment  data  are 
reported  and  discussed,  and  the  assessors  agree  on  the  evaluation 
of  the  dimensions  and  any  overall  evaluation  that  is  made. 

It  is  important  to  emphasize  that  an  assessment  center  is  a 
method  and  not  a  place  (9yham  and  iUettengel,  1974:353).  In  the 
"typical  '  assessment  center  six  candidates  come  together  for  one  to 
three  days.  The  length  will  depend  on  the  purpose  of  the  center  and 
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the  numder  of  exercises.  The  candidates  will  participate  in  exercises 
designed  to  bring  out  behavior  related  to  certain  dimensions  or 
character  traits  which  have  been  identified  by  previous  research  as 
important  to  success  in  the  target  job.  The  exercises  will  include 
personal  interviews,  job  simulations,  leaderless  group  discussions, 
and  both  oral  and  written  communication  exercises. 

As  the  candidates  perform  the  exercises  they  are  observed  by 
three  assessors  who  are  usually  higher-level  managers  and  have  received 
special  training.  The  schedule  is  arranged  so  that  assessors  see 
different  candidates  in  each  exercise  and  all  assessors  see  each  can¬ 
didate  at  least  once.  The  assessors  record  their  observation  of  a 
candidate's  behavior  on  a  special  report  form.  At  the  conclusion  of  the 
center,  the  assessors  meet  togetner  to  discuss  tne  recorded  observations 
for  each  candidate.  Based  an  their  discussions,  the  assessors  reach  a 
consensus  agreement  on  an  evaluation  of  the  candidate’s  strengths  and 
weaknesses  in  the  identified  dimensions.  A  consensus  is  also  reached 
an  the  candidate's  overall  performance.  The  assessors  then  prepare 
final  written  reports  on  each  candidate.  The  final  report  will  be 
given  to  the  candidate,  management,  or  both.  Finally,  some  type  of 
oral  feedback  is  usually  given  to  both  the  candidate  and  management. 

The  exact  design  and  operation  of  a  given  assessment  center  will  vary. 

Wore  detailed  information  on  assessment  centers  is  included  in  Chapter  IV. 

This  thesis  investigates  the  use  of  the  assessment  center  as  a 
means  of  providing  feedback  to  assist  Air  Force  officers  in  their 
management  development.  An  actual  assessment  center  is  examined  and 
considered  for  such  application.  The  center  studied  was  conducted  at 
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Ulr ight-Patterson  AFB,  Ohio  in  1074  and  1975  and  was  called  the 
Aeronautical  Systems  Qivision  (ASD)  Management  Assessment  Center.  This 
center  was  designed  to  evaluate  Air  Force  civilian  scientists  and 
engineers  for  First  level  supervisory  positions.  Detailed  objectives 
of  this  study  effort  are  outlined  below. 

Statement  of  Objectives 

The  overall  objective  of  this  thesis  is  to  investigate  the 
concept  of  assessment  centers  as  a  method  for  officer  development. 
Specifically,  the  objectives  are: 

1.  Examine  the  feasibility  of  using  an  assessment 
center  for  the  purpose  of  officer  development. 

2.  Conduct  a  validity  study  of  the  ASC  Management 
Assessment  Center,  focusing  on  both  content  and 
empirical  validity. 

3.  Outline  an  approach  for  conducting  a  pilot 
assessment  center  for  management  development 
of  Air  Force  officers. 


Scope  and  Limitations 

While  the  assessment  center  approach  to  management  development 
should  be  considered  for  all  Air  Force  officers,  this  research  was 


based  on  data  gathered  on  scientific  and  engineering  personnel.  The 


data  was  based  on  the  ASD  Management  Assessment  Center  co  iducted  ir 
1974  and  1975  for  G3-9  through  65-14  level  civilian  personnel.  This 
thesis  effort  will  provide  the  background  for  conducting  a  oilot 
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assessment  center  for  Air  Force  officers  of  comparable  grade  assigned 
to  A5D.  The  results  may  then  be  compared  to  those  obtained  with 
civilian  participants. 

Finally,  no  attempt  is  made  tc  include  a  study  on  cost  effec¬ 
tiveness  or  means  of  implementation  on  on  Air  Force  wide  basis. 

Summary 

This  initial  chapter  has  provided  an  introduction  to  the 
research  subject,  identified  the  problem,  and  outlined  the  objectives 
and  limitations  of  the  study.  The  next  chapter  details  the  method¬ 
ology  used  in  accomplish  mg  the  objectives. 
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chapter  ii 


methodology 

The  methodology  used  will  parallel  the  previously  stated 
objectives.  The  overall  methodology  is  described  below. 

1.  Conduct  a  literature  review  on  present  officer  development 
programs  and  determine  the  criteria  presently  used  in 
officer  development. 

2.  Conduct  a  literature  review  on  the  assessment  center  process, 
including  its  use  for  management  development. 

3.  Conduct  a  validity  study  on  the  ADD  Management  Assessment 
Center. 

4.  Outline  an  assessor  training  program,  including  expanded 
dimension  definitions  and  benchmarks,  for  use  in  a  pilot 
assessment  center. 

5.  Oiscuss  past-assessment  feedback.  Prepare  an  outline  for 
the  feedback  interview. 

6.  Oescribe  future  data  gathering  efforts  needed  to  further 
evaluate  the  effectiveness  of  the  assessment  c  nter  for 
management  development  of  Air  force  officers. 

Literature  Review;  Officer  Development 

Since  this  thesis  was  concerned  with  an  examination  of  the 
management  development  process  for  Air  Force  officers,  a  background  on 
the  present  methods  for  officer  development  was  needed.  This  background 
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was  included  (1j  to  provide  information  on  the  management  skills 
stressed  and  (2)  to  determine  what  deficiencies,  if  any,  exist  within 
the  present  system. 

A  review  of  the  literature  indicated  that  management  develop¬ 
ment  methods  fell  into  two  broad  categories  --  formal  off-the-job 
training  and  informal  on-the-job  training.  A  discussion  of  both  these 
areas  is  included  as  well  as  a  discussion  of  the  precommissioninc 
programs  providing  initial  management  development  training. 

Literature  Reviews  Assessment  Center 
■  e  this  thesis  integrates  the  two  processes  of  development 
arK  ..nt,  it  was  also  necessary  to  review  the  literature  per¬ 

taining  to  assessment  centers.  In  reviewing  the  literature  on  assess¬ 
ment  centers,  five  general  areas  were  considered. 

First,  information  was  sought  on  the  evolution  of  the  assess¬ 
ment  center.  Before  discussing  the  application  of  this  technique,  it 
was  necessary  to  establish  an  historical  perspective. 

Second,  the  literature  was  reviewed  to  determine  the  various 
uses  of  the  assessment  center.  Althougn  this  thesis  advocates  the  use 
for  officer  development,  there  are  other  uses  of  which  one  should  be 
aware.  It  is  often  the  case  that  a  given  assessment  center  will  serve 
more  than  one  purpose. 

Third,  the  many  considerations  involved  in  the  design  of  an 
assessment  center  were  reviewed.  Before  beginning  any  new  design 
effort,  it  is  important  to  have  a  design  checklist.  The  literature 
review  will  provide  such  a  guide  to  be  followed  in  establishing 
assessment  centers  for  management  development  of  Air  Force  officers. 
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The  fourth  issue  reviewed  was  that  of  validity  "Unlike  many 
other  management  development  techniques  .  .  .  the  assessi.  _  method  has 
teen  well  received  partly  because  properly  controlled  research  has 
shown  it  to  be  of  value"  (Byham,  1970s 153).  Even  though  the  assess¬ 
ment  center  has  proven  to  be  valid  in  general,  the  validity  of  a  given 
assessment  center  must  stand  alone.  The  literature  was  reviewed  to 
determine  now  to  go  abouc  conducting  a  validity  study. 

finally,  the  literature  was  reviewed  to  investigate  the  use  of 
assessment  centers  in  the  military.  Articles  reporting  on  both  the 
applicability  of  the  assessment  process  and  actual  military  assessment 
centers  were  reviewed. 

Several  sources  were  used  in  the  literature  reviews  on  both 
officer  development  and  assessment  centers.  The  sources  used  included 
the  AFIT  library  system,  Wright  State  University  library,  Defense 
Documentation  Center,  Air  University  library,  and  files  in  the  ASD 
Human  Resources  Center. 

Validation 

After  showing  that  the  assessment  center  is  applicable  to 
management  development  of  Air  Force  officers,  the  next  part  of  the 
methodology  was  to  introduce  an  assessment  center  design  and  to  show 
the  validity  of  that  design.  The  design  selected  was  the  management 
Assessment  Center  operated  at  A50  in  1974  and  1975  to  select  first- 
level  civilian  supervisors  in  scientific  and  engineering  positions.  A 
complete  description  of  this  assessment  center  design  is  given  in 
Chapter  V. 
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This  particular  design  was  selected  for  two  reasons.  First, 
it  was  felt  that  there  was  a  large  amount  of  similarity  tetween  civilian 
and  military  jobs  in  the  scientific  and  engineering  fields  at  ASD.  This 
similarity  is  discussed  in  Chapter  VI.  The  other  reason  was  the  oppor¬ 
tunity  to  conduct  a  validity  study  using  readily  available  data  before 
testing  the  design  with  Air  Force  officers. 

The  validation  of  the  1974-75  Management  Assessment  Center  was 
done  in  two  parts  --  content  validity  and  empirical  validity.  The 
empirical  study  was  further  divided  into  predictive  and  internal  valid¬ 
ity.  Background  on  these  types  of  validity  is  given  in  Chapter  IV. 

Content  Validity 

Following  the  procedures  outlined  in  Chapter  IV,  the  content 
validity  was  first  established.  This  was  done  by  reviewing  documen¬ 
tation  on  the  managerial  assessment  center.  The  results  of  this  review 
are  included  in  Chapter  V. 


Empirical  Validity 

The  statistical  analyses  were  accomplished  using  the  Statistical 
Package  for  the  Social  Sciences  (SPSS),  (Nie,  et  al,  1975).  All  pro¬ 
grams  were  run  on  the  Aeronautical  Systems  Division's  COC  6600  computer 
at  Wright-Patterson  AFB,  Ohio. 

The  empirical  validity  consisted  of  two  parts  --  predictive 
and  internal.  Predictive  validity  was  concerned  with  identifying  the 
strength  of  the  relationship  between  the  overall  assessment  center 
rating  and  actual  managerial  success  following  the  assessment  center 


experience.  Internal  validity  was  concerned  with  a  determination  of 
the  reliability  of  the  assessment  center. 

This  section  presents  a  description  of  the  sample  population 
and  the  methodology  used  in  determining  botn  the  predictive  and  internal 
validity . 

Sample  Population.  A  total  of  2SS  civilian  personnel  of  the 
Aeronautical  Systems  Division  and  Wright  Aeronautical  Laboratories  were 
evaluated  in  1974-1975  using  the  Management  Assessment  Center.  A  total 
of  50  centers  consisting  of  six  assessees  each  were  conducted.  In  all 
cases,  two  centers  were  run  concurrently.  All  the  assessees  were  G5-9 
through  6S-I4  level  personnel  occupying  scientific  and  engineering 
positions.  Records  for  personnel  assessed  in  the  first  IS  centers  were 
deleted  since  these  centers  were  operated  under  a  pilot  program.  From 
the  32  operational  centers,  these  with  less  than  six  assessees  were 
deleted.  The  internal  validity  was  based  on  data  from  144  individuals 
assessed  in  24  centers. 

The  predictive  validity  was  based  on  data  from  143  individuals. 
Recurds  of  individuals  no  longer  assigned  to  Wright-Patterson  AFB  were 
deleted  from  the  32  operational  centers,  since  criterion  data  was  not 
readily  available. 

Data  on  dimension  ratings  and  final  ratings  were  taken  from 
assessor  work  sheets  contained  in  the  assessee  folders  for  each  center. 
In  coding  the  data,  missing  ratings  were  not  included.  Also,  some 
assessors  gave  tie  scores  such  as  '3/4'  •  Ties  were  broken  by  rprnrdino 
the  Lcore  closest  to  that  assessor's  final  dimension  rating  for  the 
assessee . 
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Internal  Validity.  The  determination  of  the  reliability  cf  the 


assessment  center  consisted  of  four  parts: 

1.  Examination  of  the  relations  among  the  dimensions  and 
between  the  dimensions  and  the  overall  assessment  rating, 

2.  Measurement  of  the  agreement  in  ratings  among  assessors. 

3.  Identification  of  the  nature  of  the  underlying  components 
of  the  dimensions. 

4.  An  examination  of  the  amount  of  information  used  in  deter¬ 
mining  the  overall  rating. 

The  variables  and  specific  data  analysis  for  each  of  these  parts 
follows. 

A.  Correlation  of  Dimensions:  The  correlation  among  the 
14  final  Dimension  ratings  and  between  the  14  Final  Dimension  tatings 
and  the  Overall  Assessment  rating  were  accomplished  using  the  SPSS 
subprogram  PEARSON  CORR .  The  variables  used  were  the  final  ratings  for 
each  dimension  and  the  overall  rating  decemined  during  the  consensus 
meeting . 

The  correlations  of  the  dimension  ratings  for  the  six  exercises 
with  (1)  final  ratings  on  the  same  dimension  and  (2)  the  overall  assess¬ 
ment  rating  were  also  accomplished  using  the  SPSS  subprogram  PEARSON 
CORR.  In  this  case,  the  variables  used  consisted  of  the  individual 
assessor  ratings  for  each  dimension  within  each  exercise,  the  final 
dimension  rating  and  the  overall  assessment  rating.  The  correlations 
were  determined  by  first  computing  the  correlation  coefficients  for  each 
of  the  A?  vanaoies  (individual  assessor  rating  for  each  dimension  with¬ 
in  each  exercise)  for  the  three  assessors  evaluating  the  individual. 
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The  three  correlation  coefficients  for  each  variable  were  then  averaged. 
This  was  done  for  bath  the  correlations  with  the  final  dimension  rating 
and  the  overall  assessment  rating. 

B.  Tnter-rater  reliability!  "In  an  assessment  center,  the 
best  measure  of  reliability  is  the  degree  of  agreement  among  the  three 
raters"  (Norton,  Ounne,  and  Thornton,  1980:17).  The  SPSS  subprogram 
PEARSON  CORR  was  again  used  to  calculate  the  correlation  coefficients 
for  the  variables  analyzed. 

The  variables  used  to  determine  the  inter-rater  reliability  were 
the  individual  assessor  ratings  for  each  dimension  within  each  exercise. 
In  the  context  of  the  assessment  canter,  dimensions  are  the  observable 
traits  and  characteristics  exhibited  by  an  individual  in  a  simulated 
situation  or  exercise.  Each  dimension  is  observed  and  rated  in  more 
than  one  exercise.  These  dimensional  ratings  were  obtained  from  the 
assessor  worksheet  and  represent  the  assessor’s  independent  evaluation 
of  the  candidate's  strength  in  the  particular  dimension. 

The  final  inter-rater  reliability  was  calculated  in  the  following 
four  steps: 

1.  Compute  the  three  assessor  correlation  coefficient  matrix 
for  each  dimension  in  each  exercise.  This  was  accomplished 
for  each  of  the  24  times  the  center  was  run.  Each  corre¬ 
lation  was  for  an  N  =  6. 

2.  Compute  the  average  correlation  coefficient  for  each 
dimension  in  each  exercise  frum  the  above  matrices. 

8.  Calculate  the  final  average  correlation  coetfMr-ipnf  fnr 
each  dimension  in  each  exercise  by  averaging  all  the 
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average  correlation  coefficients  from  each  of  the  24 
center  runs. 

4.  Apply  the  Spearman-Brouun  prediction  formula  to  the  final 
average  correlation  coefficients  to  determine  the  inter- 
rater  reliability  measure  for  each  dimension  in  each 
exercise. 

The  Spearman-Grown  (S-8)  prediction  formula  was  usea  to  adjust 
for  the  number  of  assessors  evaluating  each  assessee.  "The  results  of 
Remmers  and  others  lead  rather  definitely  to  the  conclusion  that,  if 
judges  are  comparable,  the  reliability  of  pcoled  judgments  increases 
directly  with  the  number  of  judges  according  to  the  S-B  formula" 
(Guilford,  1936:421). 

In  the  Spearman-Brown  prediction  formula,  A  is  the  number 

Ar1 

^  1  +  (A-I)r^ 

of  assessors  and  r^  is  the  final  average  correlation  coefficient  from 
step  3  above. 


C.  Factor  Analysis:  A  factor  analysis  using  5P55  sub¬ 
program  FACTOR  was  used  to  investigate  the  number  of  meaningful  factors 
underlying  the  assessor  ratings.  This  procedure  provides  the  capability 
to  reduce  data  and  identify  a  possible  smaller  set  of  underlying  com¬ 
ponents  to  account  for  the  observed  interrelations  of  the  dimensions 
(Nie,  et  al,  1975:469). 

T  /->  nrrn  mm  1  <  pH  f  hi  ■*  4-  fc-  ~  4-  ^  — .  —  —  _  i- 

*  i  *•  o  u  i  1  w  i.  vi  ic  cni,j.i  c  pupuiatigi  i 

were  factored  using  principal  component  analysis  and  VAR  I MAX  rotation. 
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The  numder  of  factors  extracted  uias  specified  using  an  eigenvalue  of 
1.0  as  the  cutoff  criterion. 

A  second  abjective  was  to  determine  if  the  underlying  factors 
of  dimensions,  as  identified  on  the  assessment  center  feedback  reports, 
were  identical  to  the  pattern  generated  independently  by  the  factor 
analysis.  The  feedback  report  is  a  written  discussion  of  the  assessee's 
performance  in  the  14  dimensions.  On  the  feedback  report,  these  dimen¬ 
sions  are  consolidated  in  four  management  skill  areas: 

1.  Interpersonal  skills  (leadership,  energy,  forcefulness, 
persuasiveness,  and  flexibility). 

2.  Administrative  skills  (decision  making,  problem  solving, 
organizing,  acquiring  information,  and  risk  taking). 

3.  Communication  skills  (oral  and  written). 

4.  Other  (empathy  and  stress  tolerance). 

To  accomplish  this,  final  dimensional  ratings  for  the  entire  population 
were  factored  specifying  four  factors  to  be  extracted. 

0.  Regression  analysis:  A  multiple  regression  analysis 
using  SPSS  subprogram  REGRESSION  was  accomplished  to  examine  the  relation¬ 
ship  between  the  fourteen  dimensions  and  the  final  overall  assessment 
center  rating.  The  specific  abjective  was  to  determine  the  amount  of 
information  used  in  making  the  overall  rating.  To  accomplish  this  a 
stepwise  regression  was  done  using  the  entire  population.  An  examination 
of  the  resulting  regression  equation  indicated  the  amount  of  information 
used  by  the  assessors  in  determining  one  overall  lating. 
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Predictive  Validity.  In  determining  the  predictive  validity 
of  the  management  Assessment  Center,  the  SPSS  subprogram  PEARSON  CORR 
was  used  to  provide  the  correlation  coefficients  for  the  variables 
analyzed.  The  correlation  coefficient  was  used  to  identify  the  strength 
of  association  between  a  pair  of  variables  (Nie,  et  al,  1975s 276) . 

Four  independent  variables  were  identified  as  actual  criteria 
of  managerial  success  —  present  grade,  present  salary,  most  recent 
merit  appraisal  rating  and  the  number  of  promotions  since  the  assess¬ 
ment  center.  Selection  of  criteria  was  based  on  the  availability  of 
data  from  civilian  personnel  records  and  criteria  used  in  other  studies 
(Klimoski  and  Strickland,  1977:356). 

Pilot  Assessment  Center 

After  conducting  the  validity  study,  the  final  steps  in  the 
methodology  began  with  a  comparison  between  dimensions  used  in  the  ASD 
management  Center  and  criteria  presently  used  in  officer  development. 
There  appeared  to  be  enough  similarity  to  justify  use  of  the  ASD  Manage¬ 
ment  Assessment  Center  for  management  development  of  Air  Force  officers. 
To  lay  the  groundwork  for  a  pilot  assessment  center,  the  final  three 
steps  of  the  methodology  were  necessary. 

Prior  to  conducting  a  pilot  assessment  center  for  Air  Force 
officers,  it  is  important  to  have  trained  assessors.  A  training 
schedule  was  presented  and  the  major  elements  of  the  training  were 
described.  To  aid  in  assessor  training,  expanded  definitions  for  the 
14  dimensions  were  prepared.  In  addition,  benchmarks  were  prepared  to 
assist  assessors  in  calibrating  their  dimension  ratings. 
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If  the  ASO  Management  Assessment  Center  is  to  be  used  for 
developmental  purposes,  feedback  to  the  assessees  is  also  important. 

An  approach  to  giving  feedback  uias  discussed  and  an  outline  was  pre¬ 
pared  to  guide  the  feedback  interview. 

A  considerable  amount  of  data  concerning  the  ASO  Management 
Assessment  Center  has  been  collected  and  evaluated.  The  final  step  in 
the  methodology  was  to  describe  how  this  data  base  could  be  expanded  to 
further  investigate  the  effectiveness  of  the  assessment  center  for 
management  development  of  Air  Force  officers.  The  direction  such  data 
gathering  should  fallow  is  described  in  Chapter  VI. 

Summary 

This  chapter  has  described  the  methodology  for  achieving  the 
stated  objectives.  This  methodology  includes  literature  reviews  of 
both  the  present  officer  development  system  and  the  assessment  center 
method,  a  validity  study  of  an  actual  assessment  center  and  the  intro¬ 
duction  of  an  approach  for  conducting  a  pilot  assessment  center  for 
management  development  of  Air  Force  officers.  The  next  chapter  reviews 
the  present  officer  development  system. 
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CHAPTER  III 


AIR  FORCE  OFFICER  DEVELOPMENT 

Introduction 

As  discussed  in  the  previous  chapter,  this  literature  review  is 
concerned  with  an  examination  of  the  present  officer  development  system 
and,  more  specifically,  an  examination  of  what  it  develops  and  what 
limitations  exist  within  the  system.  Much  of  the  information  is  drawn 
from  three  comprehensive  studies  previously  accomplished  (Robinson,  1974; 
Dobias,  1974;  and  Place,  197Q).  The  review  includes  both  the  formal 
educational  and  informal  on-the-job  opportunities  available  for  manage¬ 
ment  development  as  an  officer  progresses  tnrougn  an  Air  Force  career. 
Although  the  Air  Farce  has  numerous  training  programs,  only  those  pro¬ 
viding  experience  or  training  in  the  area  of  management  development  are 
detailed. 

As  was  stated  in  Chapter  I,  the  terms  "Officer  Development", 
"Career  Development"  and  "Officer  Career  Development"  are  used  inter¬ 
changeably  in  the  literature.  Officer  development  includes  both 
technical  and  management  development.  Also,  as  used  in  this  thesis, 
"leadership"  development  and  "management"  development  are  taken  to  mean 
the  same  thing. 

Since  management  development  begins  prior  to  commissioning  for 
the  majority  of  Air  Force  officers,  this  study  begins  with  a  review  of 
the  different  comm:  ssinni  nr,  nrnnrams. 
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P recommissioning  Manaqament  Development  Training 


The  three  different  methods  for  bringing  commissioned  officers 
on  active  duty  provide  a  wide  range  of  education  and  training  in  basic 
management  skills.  The  methods  includes 

1.  The  Air  Force  Academy  (AFA) 

2.  Air  Force  Reserve  Officer  Training  Corp  (AFRDTC) 

3.  Officer's  Training  School  (OTS) 

The  Air  Force  Academy  (AFA) 

At  the  Air  Force  Academy  "leadership  is  based  on  the  whole 
person  concept,  meaning  that  many  attributes  of  character,  education, 
and  professionalism  are  necessary  to  compliment  academic  education  and 
complete  preparation  For  Air  Farce  Service"  (U.S.  Air  Force  Academy 
Catalog,  19V?! 113).  Ouring  the  first  two  years,  development  is  centered 
around  the  follauiship  capabilities  of  each  cadet.  This  is  seen  as  a 
necessary  prerequisite  to  leadership  development. 

Ouring  the  final  two  years,  the  military  training  and  studies 
program  is  geared  to  developing  individual  leadership  skills  through 
expanded  responsibilities  within  the  organizational  structure  of  the 
Air  Force  Academy.  The  cadet  wing  is  organized  similarly  to  an  oper¬ 
ational  Air  Force  wing,  and  cadets  are  given  the  opportunity  to  hold 
various  operational  and  staff  positions  (U.S.  Air  Force  Academy 
Catalog,  1977). 

Ouring  the  four  year  period,  the  cadets  receive  limited  text¬ 
book  management  and  leadership  training.  "After  inspecting  the  entire 
leadership  and  management  development  program  at  the  AFA,  the  results 
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reveal  that  only  one  semester  of  basic  management  principles  and  three 
semesters  an  the  behavioral  science  aspects  of"  leadership  are  taught" 
(Place,  1978:34). 

The  Air  Force  Reserve  Officer  Training  Corp  (AFRQTC) 

The  Air  Force  Reserve  Officer  Training  Corp  (AFROTC)  is  a 
commissioning  program  conducted  at  colleges  and  universities  throughout 
the  country.  Its  primary  objective  is  to  "strengthen  each  cadets' 
potential  as  a  leader  and  a  manager"  (AFR  45-48,  1973:4).  The  AFRCTC 
program  consists  of  two  separate  courses  linked  by  a  field  exercise 
scheduled  between  the  second  and  third  years.  These  two  programs  are 
the  General  Military  Course  (GMC)  and  the  Professional  Officer 
Course  (PQC). 

"The  GMC  is  developmental  in  nature  and  is  designed  to  motivate 
and  prepare  cadets  for  entry  into  the  POC"  (AFROTC  Reg  53-2,  1977:2-1). 

It  consists  of  the  very  basics  of  leadership  training  such  as  drill  and 
ceremonies,  and  custom  and  courtesies.  Those  cadets  who  enter  the 
AFROTC  program  without  completing  GMC,  receive  this  training  at  an 
extended  field  exercise. 

"The  POC  is  designed  to  prepare  cadets  for  active  duty  as  Air 
Force  officers"  (AFROTC  Reg  53-2,  1977:2-1).  During  the  two  year  period, 
approximately  25  percent  of  the  program  is  directed  to  expanding  the 
cadets'  leadership  experiences  in  officer-type  activities  (AFROTC  Reg 
53-2,  1977).  The  leadership  activities  are  designed  to  emphasize  "the 
individual  as  a  manager  in  the  Air  Force"  (Place,  1979:37).  The  material 
covered  includes  individual  motivational  and  behavioral  processes,  lead¬ 
ership,  communications,  and  group  dynamics.  Throughout  the  program, 
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each  student  receives  a  total  of  tuio  semesters  on  basic  management 
principles  and  one  semester  on  the  behavioral  aspects  of  leadership. 

Only  three  semesters  of  the  leadership  laboratory  are  designed  to 
enhance  leadership  capabilities  (Place,  1978). 

The  Officer  Training  School  (OTS)  Program 

The  Officer  Training  School  (OTS)  program  is  designed  to  pre¬ 
pare  those  individuals  already  possessing  degrees  with  the  opportunity 
to  gain  a  commission.  The  twelve  weeks  of  instruction  is  designed  to 
provide  training  to  "meet  the  fundamental  requirements  for  newly 
commissioned  officers  in  the  Air  force"  (Place,  "'978:39). 

As  with  both  the  AFA  and  AFROTC  programs,  the  structure  at  DT5 

is  designed  to  provide  experience  in  different  . . end  arid  staff 

positions  of  the  Air  Force  organization.  Additionally,  each  candidate 
receives  seventy-eight  classroom  hours  uf  leadership  and  management 
training  which  emphasizes  typical  leadership  and  management  situations 
confronting  an  Air  Force  officer  (Place,  1978). 

One  criticism  of  the  OTS  program  is  the  lack  of  indepth  stuoy 
devoted  to  any  one  area  because  of  the  time  restrictions  (Place,  1976; 
Oobias,  1974). 

Post  Commissioning  Management.  Development  Training 
Once  commissioned,  management  development  takes  the  form  of  Doth 
formal  and  informal  "opporti ’nities"  occurring  continuously  throughout 
the  officer's  career.  The  word  "opportunities"  needs  to  be  stressed 
because  management  development  becomes  botn  a  joint  and  an  individ  >3l 
effort.  Management  development  results  from  opportunities  that  the 
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supervisor  oust  make  soma  effort  to  present,  and  the  officer  must  make 
some  effort,  to  use. 

Supervisors  are  in  the  best  position  to  determine  the  needs  of 
their  subordinates.  The  supervisor  has  “the  responsibility  for  develop¬ 
ment  actions  such  as  performance  evaluations,  job  rotation  through 
si mple-to-complex  duties,  and  recommendations  for  classification, 
training,  education  and  further  assignments"  (AFR  36-23,  1979:4-1). 

Once  the  supervisor  provides  the  opportunities  to  develop  management 
potential,  it  is  then  up  to  the  individual  to  take  advantage  of  them. 
"The  Air  force  guides  ar.d  assists  in  career  planning,  but  the  officers 
must  work  out  their  individual  problems  and  gain  the  knowledge  and 
capabilities  needed  to  advance"  (ARM  36-23,  1979:4-1).  This  is  accom¬ 
plished  through  an  integration  of  both  formal  and  informal  development 
opportunities. 

Formal  Training 

Throughout  an  officers'  career,  they  are  provided  with  oppor¬ 
tunities  for  formal  educational  training.  These  opportunities  include 
Professional  Military  Education  (PME)  programs,  t.he  Air  Force  Institute 
of  Technology,  specialized  management  programs  and  self-development. 

Professional  Military  Education.  “The  basic  PME  objective  is 
to  enhance  the  professional  military  competence  of  Air  Force  Officers 
through  a  program  of  education  designed  to  broaden  perspectives, 
increase  knowledge  and  prepare  these  officers  to  assume  higher  levels 


of  command  and  staff  auties  and  responsibilities"  I,  AFR  53-8,  1976:2-1). 
Three  levels  of  education  exist:  Squadron  Officers  School,  Air  Command 


and  Staff  College,  and  Air  War  College.  Each  level  of  education  is  to 
be  given  when  it  is  most  relevant  to  a  particular  stage  of  career 
development.  This  permits  the  officer  to  develop  expertise  from  a 
progressively  higher  level  of  experience  and  maturity  (Rogers,  1975). 

The  logic  of  the  system  is  to  have  officers  attend  school  at 
critical  paints  during  their  career.  Squadron  Officer  School  becomes 
available  at  the  point  when  an  officer  is  deciding  on  an  Air  Force 
career.  Air  Command  and  Staff  College  becomes  available  prior  to 
promotions  to  field  grade.  Air  War  College  prepares  the  officer  to 
assume  senior  field  grade  and  general  officer  responsibilities  (Grove, 
1975).  As  the  officer  progresses  in  a  given  career  and  assumes  the 
responsibilities  of  higher  levels,  the  activities  and  decisions  involve 
more  areas  outside  that  specialty.  The  PME  system  increases  the 
officer's  knowledge  and  skills  in  a  wider  perspective. 

The  Squadron  Officer  School  is  the  first  level  of  PME  for  USAF 
oFficers.  Its  mission  "is  to  prepare  lieutenants  and  captains  for 
command  and  staff  tasks  required  by  the  Air  Force,  while  providing  a 
foundation  for  further  professional  developments"  (AFR  53-8,  1976:3-1). 

Leadership  anc  management  development  as  SCS  is  presented  sepa¬ 
rately  in  the  course  curriculum.  Leadership  is  covered  in  five  phases 
and  includes: 

1.  An  examination  cf  Air  Force  standards  of  leadership, 

2.  A  basic  understanding  of  how  people  interact  with  the 
leadership  process. 

3.  An  examination  of  different  leadership  styles,  and  the 
situational  approacn  to  leadership. 
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4.  A  practical  application  of  the  formal  instruction  in 
communicative  skills,  leadership,  and  management. 

5.  A  modified  version  of  an  assessment  center  which  is  used 
to  identify  individual  interpersonal  and  administrative 
skills  (Squadron  Officer  School  Curriculum  Catalog,  1977). 

Ouring  the  leadership  block,  the  students  appear  to  have  an  excellent 
opportunity  to  examine  their  own  leadership  styles  and  reevaluate  their 
effectiveness  in  many  different  situations  (Place,  1978:44). 

The  management  block  was  designed  to  build  upon  and  extend 
instruction  in  the  leadership  block.  Its  two  phases  include: 

1.  Improving  individual  skills  and  providing  a  basis  for  under¬ 
standing  the  present  Air  Force  management  systems. 

2.  An  examination  of  Air  Force  systems  for  managing  its  human, 
financial,  and  material  resources  (Squadron  Officer  School 
Curriculum  Catalog,  1977). 

The  first  phase  emphasizes  the  behavioral  aspects  of  management,  the 
second  phase  emphasizes  "hands-on"  training  exposing  the  student  to 
different  management  systems  in  use  in  the  Air  Force. 

Officers  are  selected  for  Squadron  Officer  School  on  a  com¬ 
petitive  basis  by  a  central  selection  board  at  major  comman  level. 
Approximately  85  percent  of  all  eligible  officers  can  attend  SOS  in 
residence  (Groves,  1975).  Those  not  selected  have  the  option  of  com¬ 
pleting  SOS  by  correspondence.  Non-selection  for  50S  does  not  eliminate 
an  officer  from  attendance  at  the  next  level  of  PPOE,  Air  Command  and 
Staff  College. 
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The  Air  Command  and  Staff  College  is  the  Air  Force  PfflE  school 
at  the  intermediate  level.  "The  mission  of  Air  Command  and  Staff  College 
is  to  prepare  selected  officers  for  the  command  and  staff  duties  of 
majors  and  lieutenant  colonels"  (AFR  53-8,  1976:4-1). 

Gne  of  three  instructional  areas  is  devoted  to  command  and 
management.  This  phase  includes  study  and  analysis  Qf  management  fun¬ 
damentals,  command  and  leaders  ip,  analytical  techniques  of  management, 
and  management  resources.  Two  objectives  of  this  block  are: 

1.  Apply  field  grade  leadership  and  management  skills  in 
the  Air  Force  environments. 

2.  Apply  selective  non-quantitative  decision-making  techniques 
in  deriving  solutions  to  management  problems  (Air  Command 
and  Staff  College  Catalog,  1977). 

A  comprehensive  review  of  the  literature  provided  each 
student,  coupled  with  a  random  sampling  of  lectures  conducted 
in  the  area  .  .  .  found  that  a  definitive  explanation  of  the 
terms  'field  grade  leadership  and  management  skills”  was  not 
readily  apparent.  While  the  scope  and  depth  of  training  re¬ 
quired  for  field  grade  leadership  and  management  development 
should  be  different  than  that  conducted  at  SOS,  the  literature 
revealed  that  AC5C  is  also  employing  a  'grass  roots’  approach 
to  the  development  process  (Place,  1970:46). 

"Grass  roots"  implies  the  same  fundamental  management  skill  introduced 
in  the  Sguadron  Officer  School  curriculum.  While  AOSC  should  be  pro¬ 
viding  higher  grade  officers  with  management  skills  needed  for  higher 
level  positions,  the  program  appears  to  be  duplicating  or  reinforcing 
the  development  skills  already  acquired. 

Officers  are  selected  for  attendance  at  the  Air  Command  and 
Staff  College  by  a  central  selection  board  at  Headquarters,  USAF. 
Approximately  18  percent  of  all  eligible  officers  are  able  to  attend. 
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ACSC  in  residence  (Groves,  1975).  Those  officers  not  selected  for 
attendance  in  residence  are  encouraged  to  complete  either  the  Seminar 
Program  administered  by  ACSC  at  all  Air  force  bases  or  the  correspondence 
course  (APR  53-6,  1979). 

Air  War  College  (AWC)  is  the  senior  level  Air  Force  PIY1E  school. 
"The  mission  of  the  AUlC  is  to  prepare  senior  officers  for  high  command 
and  staff  positions"  (APR  53-8,  1976s 14-1).  Management  instruction  con¬ 
sists  of  a  command  and  management  unit  which  has  the  following  general 
objectives 

To  comprehend  factors  contributing  to  leadership,  command, 
and  management  of  human  resources  in  the  Department  of  Defense; 
analytical  techniques  for  decision  making;  and  principle  methods 
used  in  defense  resource  management  (Air  'liar  College  Bulletin, 

1978s 14). 

Of  the  167  hours  allocated  to  the  command  and  management  unit,  80  hours 
are  spent  on  the  fundamental  issues  and  techniques  of  leadership  and 
management  techniques;  the  remaining  time  is  devoted  to  higher  level, 
applied  learning  techniques  (Place,  1978). 

As  with  ACSC,  officers  are  selected  for  attendance  at  Air  War 
College  by  a  central  selection  board  at  Headquarters,  USAP .  Approx¬ 
imately  12  percent  of  all  eligible  officers  are  able  tQ  attend  the 
resident  AUlC  program  (Groves,  1975).  A  5eminar  Program,  Correspondence 
Group  Study  and  individual  correspondence  program  are  also  available 
for  those  officers  not  selected  to  attend  in  residence  (APR  53-8,  1976). 

An  examination  of  the  Professional  Military  Education  programs 
uncovers  two  weaknesses.  The  first  is  the  small  fraction  of  the  total 
officer  force  given  the  opportunity  to  attend  the  schools.  This  is 
particularly  true  with  ACSC  and  AWC.  Although  a  number  of  Air  Force 
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officers  are  given  the  opportunity  to  attend  equivalent  schools  of  the 
other  services,  the  percentages  increase  to  only  21  percent  for  ACSC 
or  equivalent  and  16  percent  for  AU1C  or  equivalent  (Groves,  19?5). 

Secondly,  "the  SOS,  ACSC,  and  ADUC  curriculum  for  leadership 
and  management  development  should  be  integrated  to  reduce  redundancy 
and  overlap"  (Place,  1973:66).  ffluch  of  the  fundamentals  of  leadership 
and  management  skills  are  presented  as  a  part  of  all  three  programs. 

Air  Force  Institute  of  chnoloqy .  The  Air  force  Institute  of 
Technology  (AFIT)  represents  the  second  major  source  of  management 
development  opportunities  for  Air  Force  officers.  "The  mission  of  the 
Air  Force  Institute  of  Technology  (AFIT)  is  tD  provide  education  and 
training  to  meet  Air  Force  requiremr  ;  in  scientific,  technological, 
managerial,  medical,  and  other  fieios  as  directed  by  Headquarters,  USAF" 
(Air  Farce  Institute  af  Technology  Catalog,  1979).  This  mission  is 
accomplished  through  ootn  university  level  educational  programs  and 
continuing  education  programs.  The  university  level  work  is  designed 
to  provide  officers  with  a  "broad  educational  background  that  will 
equip  them  both  to  understand  their  cultural  and  technological  envi¬ 
ronment  and  to  analyze  and  attempt  to  solve  these  problems"  (Air  Force 
Institute  of  Technology  Catalog,  1979:2).  The  continuing  education 
programs  are  short  courses  designed  to  provide  specialized  training  to 
meet  specific  Air  Force  and  000  needs. 

Of  the  three  resident  schools,  the  School  of  Systems  and 
Logistics  offers  Liie  Giaduate  management  Programs  concerned  with 
management  development.  The  stated  purpose  of  these  programs  is  "to 
provide  selected  graduate  military  and  civilian  managers  with  an 
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educational  experience  designed  to  enhance  their  ability  to  effectively 
analyze,  design,  and  manage  complex  defense  systems"  (Air  Force  Institute 
of  Technology  Catalog,  1979 Jill).  The  graduate  programs  are  designed 
to  help  students  accomplish  the  following  objectives  related  to  manage¬ 
ment  development! 

1.  Oevelop  the  ability  to  demonstrate  the  logic,  objectives, 
and  soundness  of  their  decision-making  processes. 

2.  Analyze  complex  problems,  assess  alternatives  and  develop 
and  apply  appropriate  decision  criteria. 

3.  Communicate  effectively. 

4.  Deal  successfully  with  human,  material,  and  financial 
resources. 

5.  Increase  their  knowledge  of  management  information  systems 
and  the  capabilities  and  limitations  of  the  computer  as  a 
tool  for  managerial  decision-making  (Air  Force  Institute  of 
Technology  Catalog,  1979). 

The  continuing  education  program  offers  44  courses,  ranging  in 
length  from  one  to  ten  weeks  to  professional  logisticians  and  systems 
managers.  These  courses  are  continuously  updated  to  meet  the  changing 
needs  of  the  Air  Force  and  the  000  (Air  Force  Institute  of  Technology 
Catalog,  1979).  The  majority  of  these  courses  provide  some  management 
experience  in  specialized  fields  such  as  maintenance,  logistics  and 
material  management. 

As  with  the  PfflC  programs,  the  selection  process  for  ART  is 
competitive.  Officers  desiring  admission  to  resident  graduate  degree 
programs  request  a  letter  of  evaluation.  In  coordination  with  the 
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major  commands,  AFIYIPC  determines  the  applicants  availability  and  selects 
the  officers  for  attendance.  Again,  as  with  PME,  one  weakness  of  AFIT 
is  the  number  of  officers  who  have  the  opportunity  to  attend  the 


programs. 

Specialized  Leadership  and  management  Training.  In  addition  to 
PME  and  AFIT,  the  Leadership  and  Management  Development  Center  (LMOC) 
offers  a  number  of  programs  to  prepare  mid-  and  upoer-level  managers 
for  increased  responsibilities.  Three  programs  offered  include: 
Professional  Personnel  Management  Course,  USAF  Commander's  Course,  and 
Base  Commander's  Management  Course.  The  last  two  are  geared  specifically 
for  wing/group  or  base  commanders  and  therefore  will  not  be  discussed. 

The  Professional  Personnel  Management  Course  is  designed  to 
"contribute  to  the  professional  development  of  senior  personnel  managers" 
(AFffl  50-5,  1979:4-5.).  This  five  week  course  provides  extensive  instruc¬ 
tion  in  such  areas  as  the  management  process,  management  philosophy, 
problem  analysis,  decision  making,  management  by  objective,  organizational 
development  theory,  and  communication  analysis.  It  is  available  only  to 
USAF  Majors  through  Colonel,  occupying  key  personnel  positions  at  base, 
MAJCOM,  or  headquarters,  USAF  level  (AFM  50-5,  1979). 

Based  on  critiques,  questionnaires,  and  surveys  administered  to 
graduates  of  the  course,  the  overall  effectiveness  of  the  course  has  been 
favorable  (Place,  1975).  As  with  PME  and  AFIT,  quotas  are  small. 

A  review  of  the  objectives  of  the  different  PME  schools,  AFIT 
programs,  and  specialized  management  development  courses  indicates  some 
common  developmental  criteria.  Communication  skills,  both  oral  and 
written,  problem  solving,  leadership,  decision  making,  and  resource 


management  are  all  stressed  by  the  management  development  opportunities 
presented.  Additionally,  all  the  programs  reviewed  provide  a  back¬ 
ground  in  basic  management  theory  and  the  behavioral  aspects  of  leader¬ 
ship.  The  primary  weakness  common  tu  these  programs  is  their  limited 
availability.  Those  officers  who  are  not  given  the  formal  training 
opportunities  must  rely  on  self-development  and  informal  opportunities 
for  management  development. 

Self-development.  A  number  of  programs  exist  which  provide 
individual  officers  with  the  means  to  extend  their  management  develop¬ 
ment  experience.  The  effectiveness  of  these  programs  is  highly  depen¬ 
dent  upon  individual  effort. 

The  Extension  Course  Institute  (EC I )  offers  correspondence 
courses  in  a  wide  variety  of  subjects.  The  correspondence  courses  for 
the  Professional  military  Education  Program  are  the  primary  source  for 
management  development. 

Off-duty  education  orograms  are  offered  at  most  Air  Force  bases 
and  are  encouraged  by  Air  Force  supervisors,  management  development  is 
somewhat  limited  by  the  choices  of  courses  available  at  the  particular 
Air  Force  base.  Additionally,  these  programs  are  not  offered  at  smaller 
Air  Force  installations. 

Informal  Management  Development  Training 

Informal  management  development  is  chat  training  and  experience 
that  results  from  on-the-job  involvement  in  the  management  process 
(Oobias,  197A).  "Everything  that  goes  on  in  the  organization  will 
probably  effect  the  growth  and  development  of  managers  in  some  way 
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and  must  be  considered  in  the  broadest  sense  a  part  of  management 
development"  (Tannehill,  1970: 10). 

The  key  to  the  effectiveness  of  on-the-job  training  for  the 
officer  lies  with  the  immediate  supervisor.  Because  of  this,  both  AFiYl 
25-1  and  APR  36-23  charge  the  supervisor  with  specific  responsibilities 
concerning  the  development  of  their  subordinates. 

managers  have  the  responsibility  to  give  subordinates  (1)  infor¬ 
mal  training  in  managing  operational  systems,  (2)  assistance  in  devel¬ 
oping  and  understanding  the  skills  necessary  to  manage  both  the  human 
and  procedural  aspects  of  operations,  and  (3)  assistance  in  developing 
a  capability  to  manage  competently  their  own  components  of  the  system. 
These  responsibilities  are  to  be  fulfilled  primarily  through  on-the-job 
coaching  of  subordinates  (Arm  25-1,  1974). 

Coaching.  Coaching  is  an  on-the-job  development  tool  which 
centers  primarily  on  the  subordinate's  work.  Each  supervisor  is  required 
to  guide  the  subordinate's  development  based  on  prior  experience.  This 
includes  establishing  performance  standards,  regular  discussion  of  day- 
to-day  problems,  and  helping  subordinate's  develop  management  skills 
(Robinson,  1974),  Because  coaching  is  work-centered,  it  is  intended  to 
focus  on  the  subordinate's  available  skills  and  use  work-related  problems 
to  increase  these  skills  and  improve  understanding  of  methods  to  resolve 
future  problems  (Oobias,  1974). 

Counseling.  AFR  36-23  provides  more  specific  guidelines  for 
supervisors.  According  to  Table  4-1,  AFR  36-23,  the  supervisor  is 
required  to  continuously  assess  the  subordinate's  potential  and  develop 
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it  through  effective  leadership  and  counsel,  and  to  provide  timely 
counsel  regarding  performance  deficiencies  and  means  for  correction. 

The  Air  force  defines  counseling  as  "the  act  of  communicating 
advice,  instruction  or  judgment  to  influence  a  person’s  attitude  or 
behavior"  (ATM  35-16,  1979,  5-4).  Whereas  coaching  refers  to  the 
subordinate's  work-related  experiences,  counseling  focuses  on  the 
subordinate's  behavior  in  a  broader,  more  fundamental  sense.  It  refers 
to  specific  objectives  and  attitudes  rather  than  the  mechanics  of  the 
tasks  performed.  An  effective  counselor  can  substantially  enhance 
management  development  by  positively  motivating  the  officer  toward  the 
development  process  (Oobias,  1974). 

A  major  aspect  of  the  counseling  process  is  the  providing  of 
feedback  to  the  officer  as  a  means  to  improve  present  performance  IAFR 
36-10,  1978  and  AFM  36-23,  1979).  "ITlore  and  more  evidence  points 
toward  feedback  as  a  key  variable  in  effective  management  development” 
(Schwendimen  and  Albertus,  1977:42).  Both  AFR  36-10  and  36-23  place  the 
burden  of  responsibility  far  providing  this  feedback  on  the  supervisor. 
AFR  36-10  identifies  two  types  of  counseling,  continuing  and  periodic. 
Continuing  counseling  should  be  performed  on  a  day-to-day  basis  as  the 
need  arises.  Periodic  counseling  represents  those  sessions  scheduled  at 
regular  intervals,  such  as  every  six  months.  AFR  36-23  also  provides 
guidelines  for  when  counseling  should  be  accomplished,  for  example, 
prior  to  the  preparation  of  an  OCR,  promotion,  attendance  at  school  and 
tender  of  a  regular  commission. 

Personal  experience  and  the  results  of  the  duality  of  Life 
Survey,  Third  Edition,  (McNichols,  1980),  indicate  a  weakness  exists 
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in  both  the  coaching  and  counseling  methods  of"  informal  management 
development  for  Air  Force  officers.  The  cummulative  experience  of  two 
officers  with  16  years  service,  in  six  assignments  within  four  major 
commands,  under  14  different  supervisors  clearly  points  out  a  deficiency 
in  this  area.  Very  little  coaching  is  being  accomplished,  and  any  feed¬ 
back  about  job  performance  must  be  requested.  The  results  of  the 
Quality  of  Life  Survey  indicate  that  a  majority  of  Air  Force  officers 
are  having  the  same  experience  with  the  informal  on-the-job  training 
methods.  According  to  the  survey,  61.8  percent  of  the  officers  surveyed 
agreed  that  more  supervision  of  member  performance  and  behavior  is 
needed  at  lower  levels  within  the  Air  Force.  Additionally,  38.4  percent 
indicated  they  seldom  or  never  received  feedback  from  their  supervisor 
about  their  job  performance. 

It  seems  clear  that  a  very  key  element  of  feedback  on  management 
development  occurs  in  the  form  of  the  officer's  completed  QER.  This 
normal  occurrence  is  contrary  to  the  purpose  of  the  Officer  Evaluation 
System.  "Evaluation  reports  are  designed  for  the  purpose  of  personnel 
management  of  Air  force  officers.  They  are  not  to  be  used  as  counseling 
devices"  (AFR  36-10,  1978* 1-2 ) - 

Whatever  the  stated  purpose  of  the  OER,  it  does  represent  the 
supervisor's  assessment  of  the  subordinate.  How  valid  is  the  Officer 
Evaluation  System,  or  any  personal  appraisal  system,  for  providing  an 
accurate  assessment  of  the  officer's  management  capabilities?  Sup¬ 
ervisory  judgments  often  fail  in  their  potential  as  a  useful  source  of 
information  oecause  supervisor's  reported  judgments  cue  afTKcieu  not  only 
by  the  subordinate's  performance,  but  also  by:  (1)  lack  of  training  of 


35 


supervisors  in  he  *■ o  make  required  judgments,  (2)  poorly  designed 
rating  systems  which  cause  judgment  difficulties,  (3)  negative  feed¬ 
back  effects  which  arise  because  the  supervisor  makes  valid  judgments 
and  rates  some  subordinates  lower  than  others,  and  [4)  task  interference 
effects  which  may  result  because  of  limited  time  or  a  large  workload, 
because  the  supervisor  is  unable  to  actually  observe  the  ratee's  per¬ 
formance,  or  because  the  supervisor  lacks  the  technical  knowledge 
needed  to  judge  the  ratee's  potential  (Rummler,  1972). 

AFR  36-10,  Officer  Evaluation  System,  states  three  specific 
problems  facing  the  present  Officer  Evaluation  System- 

1 .  A  tendency  to  give  "good"  ratings  and  a  reluctance  to  give 
"low"  ratings. 

2.  A  tendency  to  rate  according  to  a  general  impression  of 
the  officer  concerned,  often  referred  to  as  the  "halo" 
effect. 

3.  Inconsistency  in  ratings  due  to  evaluation  differences  in 
understanding  the  meaning  of  the  various  characteristics 
being  rated,  and  variations  in  evaluation  standards. 

Finally,  Fournies  (1974? 20)  states  five  reasons  why  any  manage¬ 
ment  appraisal  system  fails  as  a  tool  for  management  development  and 
p-ovides  the  following  summary: 

All  these  reasons  relate  to  nonfunctionalism:  that  is  the 
forms,  policies,  manuals,  and  procedures  that  comprise  the 
program  matrix  not  only  fail  as  functional  tools,  but  become 
the  major  obstacles  to  answering  the  needs  for  which  the  pro¬ 
gram  was  created. 

The  conclusion  drawn  is  that  tne  OER,  which  appears  to  be  the  primary 
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source  of  feedback  on  job  performance,  is  often  not  a  valid  assess¬ 
ment  of  the  officer’s  capabilities. 

Job  Rotation.  One  of  the  most  generally  accepted  and  widely 
used  management  development  methods  is  joo  rotation.  "The  organization 
must  provide,  in  a  formal  way,  movement  of  managers  in  and  out  of 
learning  jobs  (a  job  that  provides  ample  decision-making  experiences 
sufficiently  different  from  those  currently  assigned)  as  a  p3rt  of  a 
total  system  aevelopment  process"  (English  and  Marchione,  1977:68). 

Job  rotation  is  the  movement  of  officers  from  job  to  job  for 
training  purposes  as  well  as  to  meet  the  needs  of  the  Air  i-c'ce.  "Varied 
assignments  provide  the  opportunity  to  gain  new  experience  and  tG  apply 
past  training"  (APR  36-23,  1979:4-1).  Each  assignment  should  be  planned 
to  contribute  further  to  the  officers  present  knowledge  of  the  Air  Force 
and  to  better  prepare  him  or  her  for  jobs  of  increasing  responsibility 
and  scope  (AFR  36-23,  1979:4-1).  Headquarters,  USAF  and  AFIY1PC  have  been 
given  the  responsibility  to  manage  the  officer  force  to  meet  the  follow¬ 
ing  assignment  goals:  to  fulfill  present  and  projected  authorizations, 
manage  available  personnel  resources  at  the  lowest  cost,  meet  mission 
requirements,  and  provide  full  career  progression  opportunities.  A 
career  monitor  at  AFffiPC  is  assigned  to  a  given  utilization  field  for  the 
purpose  of  tracking  officers  career  progression  within  that  AFSC.  One 
of  their  goals  is  to  insure  that  officers  are  given  the  best  opportunities 
to  gain  the  varied  experience  needed  for  career  progression  ano  manage¬ 
ment  development.  The  major  restrictions  with  this  development  method 
are  the  availability  of  assignments  and  the  requirement  to  meet  Air 
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Force  needs. 


Summar 


In  summary,  this  chapter  has  presented  a  review  of  the  major 
formal  and  informal  management  development  opportunities  available  to 
Air  Force  officers.  Formal  management  development  includes  the  manage¬ 
ment  education  received  during  the  commissioning  process,  in  the  Air 
Force  professional  military  education  system,  in  the  Air  Force  Institute 
of  Technology  programs,  in  specialized  management  development  courses, 
and  the  results  of  the  officer's  individual  efforts  at  se l f-beveiopment , 

A  review  of  the  objectives  of  these  programs  indicates  the  following 
common  developmental  criteria:  communicative  skills,  both  oral  and 
written,  problem  solving,  leadership,  decision  making,  a no  resource 
management. 

Informal  management  development  is  achieved  primarily  through 
on-the-job  coaching,  counseling  and  rotation  of  assignments.  A  very 
important  aspect  of  the  informal  management  development  methods  is  the 
feedback  on  present  job  performance  provided  by  the  individual's 
supervisor . 

Tnis  review  has  indicated  there  are  limitations  in  the  number 
of  officer?  r.i'le  to  receive  management  development  through  the  formal 
training  opportunities.  Therefore,  the  Air  Force  emphasis  on  manage¬ 
ment  development  for  the  majority  of  Air  Force  officers  is  on  informal 
on-the-job  development  or  self-development.  Indications  are  that  infor¬ 
mal  management  development  is  not  being  implemented  to  the  extent 
intended  by  Air  Force  policy.  This  deficiency  is  especially  true  of 
counseling  upon  which  the  orficer  can  base  furtner  management  development. 
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A  technique  which  has  potential  for  use  in  management  develop¬ 


ment  is  the 
overview  of 


assessment  center  method.  The  next  noapt-vr  prsr’r' s  an 
the  assessment  centar  process,  includin'-  its  use  as  a 


development  tool 


CHAPTER  IV 


THE  ASSESSMENT  CENTER  METHOD 

Intraduction 

There  is  an  abundance  of  material  written  on  assessment  centers. 
Over  fifty  articles  from  periodicals,  books,  research  reports,  and 
symposia  were  reviewed.  No  attempt  is  made  here  to  cover  that  material 
in  great  depth,  but  rather  an  overview  is  given  in  the  following  areas: 
evolution,  uses,  design  considerations,  validity,  and  military  appli¬ 
cations. 


E  volution 

Assessment  procedures  were  first  used  on  a  large  scale  by  German 
military  psychologists  to  select  officers  during  World  War  I  (GS5,  1946:1). 
In  1942  the  British  formed  the  War  Office  Selection  Board  (UiOSB)  to 
select  officers  far  the  British  Army.  They  patterned  the  WQSB  after 
the  German  procedures  and  devised  additional  assessment  techniques  as 
well,  3uch  as  the  leaderisss  group  discussion  (Muck,  1977:263). 

The  first  use  of  the  assessment  center  in  the  i'nited  States  was 
by  the  Office  of  Strategic  Service  (OSS)  —  set  up  to  evaluate  the 
qualifications  of  candidates  to  serve  as  overseas  intelligence  agents 
during  World  War  II  (MacKinnon,  1975:1).  The  final  report  of  the  0S5 
Assessment  Staff  was  puolished  in  a  book  entitled  Assessment  of  Men 
(055,  1948).  This  report  generated  considerable  interest  in  assessment 
center  techniques,  largely  because  of  the  contrast  with  traditional 
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paper-and-pencil  selection  methods  (Bray,  1976:16-2).  The  enthusiasm 
was  dampened  somewhat  however,  by  disappointing  validity  studies  con¬ 
ducted  by  the  OSS  Staff  (MacKinnon,  1975:2). 

The  assessment  center  method  was  first  applied  to  American 
industry  in  1956  when  the  American  Telephone  and  Telegraph  Company 
(AT&T)  began  the  Management  Progress  Study,  a  longitudinal  research 
program  designed  to  follow  the  growth  and  development  of  a  sample  of 
young  managers  (Cohen,  Moses,  and  Byham,  1974:6).  The  first  AT&T  assess¬ 
ment  center  was  fashioned  after  the  OSS  Center  and  staffed  by  profes¬ 
sional  psychologists  (Bray,  1976:16-2).  A  total  of  274  managers  were 
assessed  and  then  reassessed  eight  years  later  (Bray,  Campbell,  and 
Grant,  1974:8).  Results  cf  the  study  were  not  revealed  to  management, 
but  were  used  far  research  purposes  only. 

The  first  operational  assessment  center  was  established  at 
Michigan  Bell  in  1958  for  use  in  the  selection  of  entry-level  managers 
(Cohen,  Moses,  and  Byham,  1974:6).  The  establishment  of  this  center 
marked  the  beginning  of  a  new  era  for  assessment  centers,  since  it  was 
staffed  entirely  by  laypersons  and  not  dependent  on  professional  assessors 
(Bray,  1976:16-2).  In  the  early  1960's  several  other  companies  began 
using  assessment  centers.  These  included  Standard  Oil  of  Ohio,  IBM, 

General  Electric,  Sears,  Wickes,  and  J.C.  Penny  (MacKinnon,  1975:3) 

Since  the  OSS,  the  first  major  use  of  the  assessment  center 
method  in  the  Federal  Government  was  by  the  Internal  Revenue  Service  (IRS). 
The  IRS  program  first  started  in  1959  to  identify  first-level  supervisors. 
Since  tnen,  it  nas  oeen  expanded  to  select  candidates  for  an  Executive 
Development  Program.  Other  federal  agencies  using  assessment  centers 
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include  the  C*vil  Service  Commission,  Office  of  management  and  Budget, 
federal  Aviation  Administration,  Social  Security  Administration,  Depart¬ 
ment  of  Housing  and  Urban  Development,  the  U.S.  Army,  and  the  U.5.  Air 
Force  (Byham  and  UUettengel,  1974).  State  governments,  as  well  as 
foreign  governments  have  also  implemented  assessment  centers.  8yham 
and  Wettengel  (1974)  provide  an  excellent  overview  on  government  usage 
of  assessment  centers. 

MacKinnon  (I975s5)  states  that  the  rapid  multiplication  of 
assessment  centers  is  largely  due  to  the  establishment  of  managerial 
consulting  companies  that  assist  clients  in  setting  up  assessment  centers. 
Three  of  these  firms  are  the  American  Management  Association,  Develop¬ 
ment  Dimensions,  and  Assessment  Oesigns.  In  1980,  over  2000  organi¬ 
zations  are  using  assessment  centers  (Curran,  1980). 

Uses  of  Assessment  Centers 

Just  as  there  are  many  organizations  using  assessment  centers, 
they  are  used  for  many  purposes.  In  reviewing  the  literature,  the 
following  uses  were  found!  employment,  early  identification,  placement, 
advancement,  development,  affirmative  action,  and  training.  Each  of 
these  uses  is  commented  on  below. 

Only  a  few  examples  were  found  of  organizations  which  use  the 
assessment  center  to  aid  in  employment  decisions.  Usually,  such  appli¬ 
cations  involved  recent  college  graduates  applying  for  management  jobs 
or  individuals  applying  for  sales  jobs.  "5everal  Bell  5ystems  telephone 
companies  as  well  as  Sears,  Roebuck  have  used  assessment  centers  in  the 
college  employment  process"  (Bray,  1976:16-8).  The  U.5.  Army  has  also 


used  assessment  centers  in  a  similar  capacity  as  precommissioning 
screening  devices  (Veaudry  and  Campbell,  1974:24).  bray  (1976:16-6) 
notes  that  both  candidates  for  management  and  sales  jobs  have  "shown 
a  clear  willingness  to  be  assessed,  even  when  their  own  time  is  involved." 

The  assessment  center  has  also  been  used  to  help  identify  people 
with  management  potential  early  in  their  careers  (Kraut,  1S76:3l). 

"There  are  many  situations  in  which  management  potential  must  be  iden¬ 
tified  at  an  early  stage  so  various  administrative  aciions  can  be  taken" 
(Byham,  1971:11).  Bray  (1976:16-7)  summarizes  the  use  for  early  iden¬ 
tification  as  follows: 

The  purpose  of  the  assessment  is  not  to  render  a  final 
judgment  of  the  candidate's  promotability  to  management  levels, 
but  rather  to  identify  those  who  have  promise  for  the  future. 

The  intent  is  to  give  those  with  high  potential  special  devel¬ 
opment  opportunities  and  accelerated  treatment  so  that  they  can 
reach  target  positions  significantly  earlier  than  might  other¬ 
wise  be  the  case. 

Placement  is  seldom  used  as  a  purpose  of  assessment.  According 
to  Bray  (1976:16-7),  this  is  most  likely  because  assessment  is  usually 
aimed  at  a  more  general  management  level  rather  than  at  specific  jobs. 
Assessment  is,  however,  often  a  factor  in  placement  of  individuals  in 
jobs  that  will  best  suit  their  talents  and  provide  development  essential 
for  a  long-term  career  (Kraut,  1976:32). 

The  most  frequent  use  of  the  assessment  center  is  for  advance¬ 
ment  or  promotion  purposes  (Bray,  1976:16-7).  Assessment  for  promotion 
occurs  at  all  levels  of  management.  "It  is  probably  most  common  at  the 
lower  levels,  but  many  organizations  restrict  tneir  assessment  to  middle 
managers"  (Bray,  1976:16-7).  many  authors  report  that  the  assessment 
center  leads  to  better  promotion  decisions  which  are  based  on  realistic 
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systemmatically  gathered  data  (Bray,  Campbell,  and  Grant,  1974;  Cohen, 
ITlosas,  and  Byham,  1974;  MacKinnon,  1975;  Kraut,  1976;  Norton,  1977; 
and  Hinrichs,  1978]. 

Although  not  usually  the  primary  purpose,  assessment  centers 
are  also  used  for  development  purposes.  In  numerous  articles,  authors 
extol  the  virtues  of  the  assessment  center  for  use  as  a  professional 
development  tool  (Byham,  1971;  Kraut,  1972;  Byham  and  Wettengel,  1974; 
Ueaudry  and  Campbell,  1974;  MacKinnon,  1975;  Bray,  1975:  Kraut,  1976: 
and  Boehm  and  Hoyle,  1977).  The  degree  to  which  a  center  is  used  for 
development  varies.  "Some  organizations  use  assessment  programs  exclu¬ 
sively  for  personal  development  in  order  to  help  people  diagnose  their 
competencies  and  to  help  improve  them"  (Kraut,  1976:32).  Preparation 
of  individual  development  plans  is  usually  a  primary  or  strong  secondary 
abjective  of  assessment  centers  only  above  the  bottom  level  of  management. 
This  is  true  because  a  large  proportion  of  those  assessed  at  the  bottom 
level  will  not  advance  to  management  (Byham,  1971:11).  Whether  or  not 
an  intended  purpose  of  an  assessment  center,  most  participants  report 
increased  self-insight  into  strengths  and  weaknesses  after  being 
assessed,  and  that  insight  is  fairly  accurate  according  to  Byham  (1971:12). 
Cray  (1976:16-7)  offers  the  following  explanation  for  why  there  are  few 
assessment  centers  for  development  only: 

It  has  proved  difficult  for  organizations  to  stick  to  a 
strict  developmental  model  in  their  assessment  activities.  Once 
extensive  evaluative  information  on  individuals  is  available,  the 
pressure  to  use  that  information  as  a  ;  ide  to  advancement  and 
placement  is  extremely  strong. 
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relationship"  between  assessment  and  development.  They  explain  this 


relationship  as  follows: 

The  ultimate  goal  of  assessment  center  programs  is  essentially 
the  same  as  the  wide  variety  of  existing  development  strategies  -- 
to  maximize  individual  effectiveness  (job  preference)  and  conse¬ 
quently  the  attainment  of  organizational  success. 

An  example  of  a  program  which  combines  assessment  ana  development  is  the 

Leadership  Development  Program  used  at  the  L).S.  Air  Force  Squadron 
1 

Officer  School.  Students  are  assessed  during  the  first  two  weeks  and 
then  given  the  opportunity  to  participate  in  a  development  program  to 
work  on  identified  weaknesses  or  other  areas  of  interest  during  the  last 
nine  weeks  (Vincent,  1980).  MacKinnon  (1975:32-33)  feels  that  the  com¬ 
bining  of  assessment  and  development  may  be  the  wave  of  the  future.  As 
an  example,  he  cites  Development  Dimensions  Inc.,  which  offers  a  full 
range  of  modules  for  the  development  of  managerial  skills,  in  conjunc¬ 
tion  with  their  assessment  services. 

A  relatively  new  purpose  of  assessment  centers  is  for  use  in 
affirmative  action  programs.  It  is  important  to  couple  such  programs 
to  early  identification  programs  in  order  to  speed  along  the  advancement 
of  minorities  and  women.  "Normal  processes  of  identification,  develop¬ 
ment,  and  advancement  are  often  lengthy,  and  it  is  particularly  necessary 
to  identify  minority-group  members  with  higher  potential  so  that  they 
can  be  moved  along  more  quickly"  (Bray,  1976:16-8).  A  good  example  may 
be  found  at  AT&T,  where  management  potential  in  blacks  and  women  is 


The  assessment  center  used  at  505  was  patterned  after  thE  A5D 
Management  Assessment  Center  operated  at  Uiright-Pat.terson  AFB,  Ohio,  in 
1974  and  1975.  It  also  contains  dimensions  usually  found  in  management 
ccccmcrs t.  centers >  such  2c  p ret i crt  sc  1 v i. c 5  f  f  ibil  ’  ty,  rnr* ^  ; ' 1 3 ^  °  • 

forcefulness,  willingness  to  lead,  etc.  For  these  reasons,  the  505 
assessment  center  is  considered  to  be  a  management  development  tool. 
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identified  during  their  second  year  of  employment  so  that  appropriate 
training  and  development  activities  can  be  planned  (Byham,  1971:11). 
Norton  and  Edinger  ( 1978:21)  explain  houi  the  Federal  Government  has 
promoted  the  use  of  the  assessment  center  for  affirmative  action: 

Federal  enforcement  agencies  such  as  the  Civil  Service 
Commission  have  encouraged  organizations  to  use  assessment 
centers.  The  fact  that  the  assessment  center  is  a  good  sample 
of  the  job  of  a  manager  may  make  it  less  subject  to  legal  chal¬ 
lenge  on  Equal  Opportunity  grounds. 

A  final  purpose  of  assessment  is  for  the  evaluation  of  training 
programs.  Steiner  (1975:237)  states  that  the  assessment  center  "can 
be  used  as  a  highly  accurate  means  for  evaluating  executive  development 
or  other  types  of  training  programs."  He  advocates  the  use  of  control 
and  experimental  groups.  The  experimental  group  would  receive  training 
while  the  control  group  would  not.  Then,  both  groups  would  be  assessed 
to  evaluate  the  effectiveness  of  the  training.  Bray  (1975:16-14)  advo¬ 
cates  a  similar  approach  and  suggests  "It  may  be  that  the  assessment 
center  approach  can  finally  throw  some  light  on  the  overall  effective¬ 
ness  of  management  training  in  pinpointing  its  strengths  and  weaknesses 

ITiacKinnon  (1975:10)  provides  an  appropriate  summary  concerning 
the  purposes  of  assessment  centers: 

Broadly  speaking,  the  purpose  of  assessment  centers  in 
business  and  industry  is  to  identify  managerial  potential  for 
purposes  of  selection,  placement,  promotion,  or  development 
(or  same  combination  of  these),  but  which  of  these  is  most 
stressed  varies  from  program  to  program. 

It  also  appears  that  assessment  centers  will  play  a  greater  role  in 

affirmative  action  and  training  evaluation  in  the  future. 


Oesign  Considerations 

Although  there  are  multiple  uses  of  assessment  centers,  and  even 
though  there  is  no  set  pattern  for  design  of  an  assessment  program, 
there  are  certain  considerations  which  must  be  taken  into  account  when 
setting  up  a  new  assessment  center.  Some  of  these  considerations  are 
highlighted  below.  A  set  of  guidelines  endorsed  in  1975  by  the  Third 
International  Congress  on  the  Assessment  Center  Method  is  included  as 
Appendix  A.  Hereafter,  any  reference  to  these  guidelines  will  be 
referred  to  as  "the  Standards". 

The  most  important  point  in  designing  an  assessment  center  is  to 
have  the  support  of  the  organization,  especially  top  management.  Also, 
regardless  of  the  purpose,  "Assessment  centers  should  be  incorporated 
as  part  of  a  total  system  rather  than  as  a  process  that  operates  in  a 
vacuum"  (Moses,  1975:4). 

Once  organizational  support  is  established  it  is  necessary  to 
define  the  target  job  for  assessment  and  to  identify  the  dimensions 
critical  to  success  in  that  job.  Bray  (1976i16-8)  describes  this 
process  as  follows: 

Methods  of  identifying  the  dimensions  to  be  used  may  Include 
job  analysis  and  descriptions,  but  they  are  most  often  based  on 
special  interviews  with  line  and  staff  managers  in  the  organi¬ 
zation  who  supervise  the  level  of  work  in  question  or  who  have 
special  staff  knowledge  thereof. 

In  order  to  defend  the  fairness  and  soundness  of  the  assessment  center 
it  is  also  important  to  maintain  a  well-documented  record  detailing  the 
process  which  leads  to  the  dimensions  being  assessed  (Byham,  1978:4). 

Few  assessment  centers  use  exactly  the  same  exercises,  however, 
all  rely  heavily  on  simulations  designed  to  elicit  multiple  dimensions 
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Prom  the  participants.  The  simulations  are  based  on  situations  the 
candidates  are  likely  to  face  in  the  target  job.  Bray  (1976:16-9) 
offers  the  following  advice: 

In  planning  the  assessment  center,  there  should  be  a  good 
amount  of  redundancy  between  exercises  so  that  a  particular 
dimension  can  be  seen  in  more  than  one  setting.  The  more  of 
this  the  batter,  since  it  adds  to  the  reliability  of  judgments 
of  the  various  dimensions. 

Exercises  common  to  most  assessment  centers  include  an  in-basket,  leader¬ 
less  groups,  and  a  management  problem.  The  content  will  differ  however, 
depending  on  the  type  job  and  management  level,  for  additional  infor¬ 
mation,  a  thorough  discussion  of  assessment  center  techniques  is  pro¬ 
vided  by  Crooks  (1977:69-07). 

Assessors  are  usually  either  professional  psychologists  or  man¬ 
agers  two  or  three  levels  above  the  candidates  and  not  in  a  supervisory 
capacity  over  them.  Assessors  should  be  carefully  selected.  "The  basic 
criteria  for  the  position  of  assessor  have  been  familiarity  with  the 
position  in  question,  a  successful  career  as  a  manager  and  possessing 
skills  much  like  the  ones  on  which  the  candidates  are  to  be  evaluated" 
(Jaffee  and  Frank,  1978:51).  "The  ratio  of  assessors  to  assessees  is 
usually  about  one  to  two  or  three"  (MacKinnon,  1975:9). 

When  nonprofessional  assessors  are  used,  they  must  be  trained 
before  the  center  begins.  "Training  time  for  assessors  can  range  from 
one  day  to  three  weeks  and  time  spent  on  an  assessment  assignment  will 
vary  from  one  week  to  one  year  in  duration"  (Jaffee  and  Frank,  1978:52)= 
There  are  many  methods  employed  to  conduct  assessor  training.  Byham 
(1970:157)  provides  the  following  information  on  assessor  training: 
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The  most  common  method  of  training  is  by  understudy.  In  the 
usual  situation,  an  assessor-in-training  sits  through  an  entire 
assessment  cycle  as  a  nonvoting  member.  Another  method  of  asses¬ 
sor  training,  particularly  when  assessment  centers  are  being 
introduced,  is  to  have  the  assessors  go  through  the  assessment 
experience  first  as  candidates.  Everything  is  the  same  except 
that  there  are  no  assessors  present.  In  a  typical  training 
situation,  the  assessors  go  through  an  activity  such  as  group 
discussion,  and  then  critique  the  discussion  and  identify 
possible  areas  of  observation  afforded  by  the  situation.  Sev¬ 
eral  companies  videotape  activities  to  give  assessors  practice 
in  making  observations. 

The  Standards  in  Appendix  A  prcvide  additional  information  on  assessor 
training  and  list  minimum  training  requirements.  Also,  the  management 
consulting  firms  such  as  Development  Dimensions,  Inc.  provide  assessor 
training  services. 

Individuals  may  become  candidates  for  assessment  either  by  self- 
nomination,  nomination  by  their  supervisor,  nr  automatically  at  certain 
career  points.  According  to  Howard  (1974i1l9),  it  is  contradictory  to 
select  candidates  based  on  their  supervisor's  nomination  "since  a  basic 
purpose  is  to  find  a  better  way  of  rating  potential  than  reliance  on  the 
supervisor's  judgment."  8asic  information  which  should  be  provided  to 
participants  is  listed  in  Appendix  A. 

Although  it  is  the  final  design  consideration  to  be  discussed 
here,  feedback  is  one  of  the  most  important  factors  in  an  assessment 
center.  Except  for  programs  which  are  purely  research  in  nature,  prac¬ 
tically  all  assessment  centers  provide  some  form  of  feedback  either  tc 
the  organization,  the  candidate,  or  both.  The  purpose  of  the  center  will 
influence  the  type  of  feedback.  A  center  used  for  management  development 
will  emphasize  future  needs  and  will  provide  feedback  to  the  candidate 
and  to  the  supervisor  .  most  cases.  Whether  or  not  the  supervisor 
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is  included  is  often  left  up  to  the  candidate  (Bray,  1976:16-10). 
MacKinnon  (1975:10-11)  describes  the  content  of  the  feedback  as  follows: 

The  content  of  the  feedback  usually  includes  a  description 
of  the  candidate's  performance  in  the  assessment  center,  an 
indication  as  to  houi  he  was  viewed  by  his  peers  as  well  as  by 
the  assessment  staff,  a  review  of  his  judged  strengths  and 
weaknesses,  and  in  some  instances,  an  outlined  plan  for  his 
future  development  as  a  manager  or  even  as  a  person.  The 
giver  of  the  feedback  is  usually  the  staff  member  who  wrote 
the  report  on  the  candidate. 

Another  type  of  feedback  is  the  written  report  prepared  for  the 
organization.  The  written  report  may  be  used  in  both  centers  designed 
for  selection  or  promotion,  as  well  as  centers  designed  for  development. 
MacKinnon  (1975:142)  also  comments  on  the  written  report: 

In  general,  reports  are  of  one  or  two  kinds,  depending  upon 
the  purpose  of  the  assessment  program.  Most  typically  they 
describe  the  performance  of  the  candidate  in  the  assessment 
center,  and  provide  ratings  of  the  candidate  on  the  rated  vari¬ 
ables,  with  an  overall  rating  of  the  candidate's  managerial 
potential  if  selected  or  promoted.  In  companies  that  use  the 
assessment  findings  in  creating  tailor-made  programs  for  the 
development  of  their  personnel  —  both  as  persons  and  as  man¬ 
agers  —  a  second  report,  or  a  second  part  of  the  single  report, 
is  concerned  with  developmental  recommendations. 

Practices  regarding  feedback  vary  from  organization  to  organi¬ 
zation.  Slivinski  and  Bouregois  (1977:143-159)  have  surveyed  a  number 
of  organizations  to  review  and  contrast  feedback  practices.  Their  con¬ 
clusions  are  as  follows: 

While  the  feedback  procedures  generally  adopted  by  most 
of  the  organizations  emphasize  both  the  selection  and  the 
developmental  aspects  of  the  assessment  center  data,  most 
organizations  are  becoming  mare  and  more  developmental  in 
their  emphasis. 

These  are  some  of  the  principal  areas  to  be  considered  when 
designing  an  assessment  center.  Although  a  new  center  can  be  patterned 
after  existing  centers,  each  application  of  the  assessment  center  must 
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be  considered  a  special  case  and  planned  accordingly.  For  additional 
information,  Jeswald  ( 1977:45-66)  provides  a  thorough  overview  on 
issues  involved  in  establishing  an  assessment  center. 

Validity 

The  literature  review  on  assessment  center  validity  is  divided 
into  three  main  area.  --  content  validity,  predictive  validity,  and 
internal  validity.  Content  validity  is  concerned  with  how  accurately 
the  assessment  center  represents  the  target  job.  Predictive  validity 
is  concerned  with  "correlations  and  other  data  showing  relationships 
between  the  evaluations  and  subsequent  ’success'  in  management"  (Huck, 
197?t262).  Internal  validity  is  concerned  with  "Factorial  results  and 
the  correlations  between  the  various  assessment  techniques  and  the 
assessment  ratings"  (Huck,  1977:264).  These  rssults  may  be  used  to 
determine  the  consistency  of  ratings  and  raters.  The  results  may  also 
lead  to  observations  concerning  relationships  among  dimensions  and 
between  dimensions  and  the  overall  reting. 

An  assessment  center  "is  validated  through  content  validity  by 
showing  that  it  is  related  in  content  to  a  significant  and  consistent 
aspect  of  successful  behavior  in  the  job  being  filled"  (Morton,  1977:445). 
The  initial  step  in  establishing  content  validity  is  to  conduct  a 
thorough  job  analysis  of  the  target  job.  This  can  be  done  either  through 
a  survey  of  successful  managers  familiar  with  that  job  or  through  a  work 
sampling  study  in  which  persons  in  the  target  job  are  observed.  As  part 
of  the  job  analysis,  the  dimensions  to  be  assessed  will  be  defined, 
unce  oimensions  are  defined*  exercises  are  selected  or  developed  which 
will  bring  out  the  desired  dimensions.  After  assessors  have  seen 
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trained  to  observe  and  record  the  identified  dimensions,  a  pilot  center 
she.1,  be  run  to  w.r*  out  any  problem  and  give  the  assessors  the  oppor¬ 
tune  ■  '  ous.  ;>'e  actual  behavior.  When  these  steps  are  carefully 

followed  an'  oncumented  an  assessment  center  may  be  said  to  have  con¬ 
tent  val-  *ty. 

Predictive  validity  studies  have  been  carried  out  by  many 
organizations.  Numerous  authors  have  published  reviews  on  such  studies 
including  Cohen,  Moses  and  8yham  (1974),  Howard  (1974),  MacKinnon 
(1975),  Huck  (1977),  Klimoski  and  Strickland  (1977),  and  Norton  and 
Edinger  (1978).  For  their  1977  article,  Klimoski  and  Strickland 
(1977:354)  reviewed  over  ninety  studies  and  found  "few  validity  studies 
per  se  being  publ  .ned  after  1  ;72."  Table  4-1,  from  their  article, 
summarizes  the  "primary  source  validity  studies  on  assr -  s,  ant  centers" 
(Klimoski  and  Strickland,  1977:355). 

The  only  major  validity  study  found  since  the  Klimoski  and 
Strickland  article  was  a  foilow-on  study  of  ISM  managers  by  Hinrichs 
(1970).  In  the  study,  47  individuals  were  assessed  in  1967  to  predict 
advancement  in  management  and  then  followed  up  eight  years  later.  This 
study  appears  to  be  the  only  "pure"  validity  study  since  the  AT&T  Manage¬ 
ment.  Progress  5tudy.  The  assessment  center  results  were  "not  used 
aaministratively  over  the  eight  years  so  the  data  were  not  contaminated" 
(Hinrichs,  1978:597).  The  follow-up  criteria  used  were  actual  position 
level  attained  and  change  in  position  level.  The  study  also  used  on 
independent  evaluation  of  management  potential  made  oy  two  managers  in 
1967.  Correlation  results  are  shown  in  Table  4-2. 


52 


TABLE  4-1 

VALIDITY  STUDIES  OF  OVERALL  ASSESSMENT  RATINGS  FROM  PUBLISHED  SOURCES 
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TABLE  4-2 


CORRELATION  RESULTS  CF  H INRICH'S  STUDY 


Predictor 

Position 

Level 

Yr  1  Yr  8 

Change  In 
Position  Level 

Yr  1  to  Yr  8 

Assessment  Center 

.26* 

.46* 

CM 

CM 

Manager 

.32* 

.55* 

.22 

n 

47 

30 

30 

*p  .05 

Hinrichs  (1978:597)  concludes  as  follows: 

While  the  manager's  prediction  based  upon  a  cartful  review 
of  personnel  records  predicts  the  8-year  criterion  more  signif¬ 
icantly  than  does  the  assessment  center,  the  shrunken  multiple 
correlation  of  the  two  together  is  .58  ....  Both  predictors 
do  significantly  and  independently  predict  the  criterion  after 
sight  years. 

The  AT&T  Management  Progress  Study  is  cited  frequently  as  the 
most  important,  most  comprehensive  and  one  of  the  only  "pure"  predictive 
validity  studies  (Cohen,  Moses,  and  Byham,  1974:10;  Howard,  1974:122; 
MacKinnon,  1975:19;  and  Norton,  1977:446).  The  correlation  of  the 
assessment  center  rating  (predicting  making  or  not  making  management) 
"with  actually  making  middle  management  was  .44  for  college  graduates 
and  .71  for  non-graduates"  (Norton,  1977:446). 

In  their  review,  Cohen,  Moses,  and  Byham  (1974:10)  summarized 
nineteen  research  studies  taken  from  a  variety  of  industrial  settings. 
They  computed  the  median  validity  coefficients  to  summarize  the  accuracy 
of  prediction.  The  median  r  was  .33  in  predicting  job  performance; 
median  r  was  .63  in  predicting  job  potential;  median  r  was  .40  in 
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predicting  job  progress;  and  the  overall  median  r  far  all  studies  was 
.37  (Cohen,  Moses,  and  Byham,  1974:21). 

In  another  study,  Norton  (1977)  compared  the  validity  of 
assessment  centers  with  traditional  methods  for  predicting  managerial 
success.  Norton  (1977:442-443)  states  the  following  conclusions: 

A  review  ar  this  literature  shows  that  the  validity  of  the 
managerial  assessment  center  as  a  predictor  of  managerial  success 
is  stable  across  different  organizations  and  different  managerial 
positions.  Comparisons  with  the  literature  on  traditional  methods 
for  predicting  managerial  success  reveals  that  the  average  valid¬ 
ity  of  the  assessment  center  is  about  as  high  as  the  maximum 
validity  attained  by  use  of  these  traditional  methods. 

A  common  problem  in  predictive  validity  studies  concerns  the 
selection  of  criteria  to  measure  management  success,.  In  discussing  this 
problem  in  their  article,  Klimoski  and  Strickland  (1977:355)  take  the 
position  that  the  frequently  used  criteria  listed  in  Table  4-1  "may 
have  less  to  da  with  managerial  effectiveness  than  managerial  adaptation 
and  survival."  Passible  reasons  given  for  use  of  these  criteria  include 
easy  access  and  ready  acceptance  by  operating  managers.  They  point  out 
that  "little  published  research  exists  comparing  alternative  predictors 
with  assessment  center  predictions"  (Klimoski  and  Strickland,  1977:357) 
As  possible  alternatives  they  suggest  biographical  data,  peer  nomi¬ 
nations,  and  training  director  or  supervisory  ratings  or  rankings. 

In  reviewing  the  literature  on  internal  validity,  most  articles 
included  inter-rater  reliability  and  factor  analysis.  Since  assessment 
centers  use  multiple  assessors,  the  issue  of  inter-rater  reliability  is 
important  because  it  is  a  measure  of  consistency  between  assessors.  A 
summary  of  inter-rater  reliability  data  published  by  Howard  (1974)  is 
shown  in  Table  4-3.  Howard  (1974:122)  concludes  that  "Inter-rater 


SumtTlARY  Of  INTER-RATER  RELIABILITY  STUDIES  OF  ASSESSMENT  PROCEDURES 


to 

m 

a 

CO 

10 

cn 

X 

C7» 

ai 

d 

^r 

OJ 

to 

• 

• 

• 

* 

cn 

i-H 

r— 

<T 

m 

Or 

• 

CJ 

/ — \ 

i 

c 

i 

i 

a 

r- 

X 

ifi 

iT) 

Is- 

o 

in 

IN 

o 

Cl 

T~ 

GJ 

to 

to 

(0 

r- 

■ 

uO 

• 

r- 

Is- 

• 

in 

• 

d 

i 

u 

QJ 

CN 

X 

to 

CD 

CXt 

cn 

cn 

m 

• 

• 

• 

• 

• 

• 

H 

T— 

o- 

CO 

cn 

X) 

cO 

CO 

•• 

•• 

•« 

•• 

in 

CM 

«• 

(n 

« 

• 

• 

• 

• 

* 

• 

•• 

in 

in 

•• 

in 

•• 

in 

•• 

in 

•• 

CO 

OD 

a 

<r 

QJ 

ra 

CO 

tn 

cn 

cn 

on 

in 

Cl 

in 

ai 

in 

cn 

in 

• 

« 

u 

r- 

m 

cn 

cn 

c 

gi 

c 

cn 

c 

CP 

c 

cn 

c 

cn 

i 

d 

c a 

-« 

CO 

•• 

CD 

•• 

— 

•• 

•• 

•• 

c 

h 

c 

■H 

c 

•w 

c 

•H 

c 

•*H 

c 

•• 

II 

c 

T— 

u 

in 

• 

in 

• 

in 

in 

in 

in 

in 

•H 

•H 

_* 

•r| 

-H 

J* 

•H 

in 

o 

i 

cn 

(X 

o 

□i 

ai 

cn 

cn 

4-J 

c 

4-J 

c 

-U 

c 

4-> 

c 

4-> 

c 

4-) 

Cl 

C 

c 

l, 

i  c 

(( 

c 

II 

c 

c 

c 

c 

c 

ca 

10 

ia 

to 

(a 

ia 

<a 

(0 

m 

(0 

to 

c 

(0 

*□ 

QJ 

•H 

*H 

•H 

••H 

•H 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

■"4 

•*H 

CN 

Li 

4-1 

4-> 

!u 

4-3 

4J 

.+J 

4-> 

73 

Is- 

10 

c 

(0 

(0 

TO 

10 

fa 

fa 

ta 

y— v 

r-v 

/— V 

<0 

QJ 

• 

3 

x 

X 

X 

X 

X 

X 

X 

m 

n 

u 

(a 

x 

c 

X 

E 

a 

in 

in 

in 

in 

in  in 

in 

in 

4-) 

4-> 

to 

4-» 

4-J 

4-J  4-> 

4-> 

cu 

in 

in 

in 

in 

tn  in 

in 

in 

■  H 

•H 

u~ 

cn 

•/H 

•H 

-H  -rH 

■iH 

in 

in 

cn 

cn 

□ 

c 

cn 

cn 

cn  cn 

CP. 

f0 

u 

a 

in 

o 

in 

in 

in 

in 

•H 

in 

o 

□ 

O  O 

o 

o 

H 

L. 

—i 

l. 

Ll 

u 

in 

-4-J 

u 

r—i 

H 

■”H  i-H 

f-H 

X 

in 

O 

0) 

□ 

OJ 

QJ 

0) 

QJ 

Li 

ia 

0J 

o 

o 

a  o 

a 

a 

in 

X 

cn 

X 

cn 

OT 

cn 

cn 

•H 

c 

> 

X 

X 

X  X 

X 

OJ 

(J 

(0 

u 

(0 

(0 

(0 

ra 

(0 

Li 

Li 

u 

u 

u  u 

u 

U 

in 

>, 

c 

>s 

c 

c 

c 

c 

a 

0J 

OJ 

>« 

>N 

X  >4 

>4 

QJ 

in 

in 

10 

cn 

(0 

10 

(0 

10 

-*J 

in 

in 

in 

in  in 

in 

ri 

cC 

a 

e 

a 

E 

E 

£ 

E 

H 

•— H 

A 

Q. 

a 

cl  a 

a 

e 

»— 4 

(0 

o 

0 

CN 

co 

CN 

tn 

ID 

in 

ID 

*x 

fN 

CN 

CN  CN 

CN 

c 

X 

X 

L- 

0) 

D 

3 

E 

D 

O 

in 

E 

C 

in 

10 

Li 

Cl 

ti 

cn 

E 

4-1 

Q 

c 

aj 

i 

GJ 

cn 

CT1 

c 

Cl 

c 

O 

in 

Li  fO 

•H 

•H 

•H 

♦h 

u 

QJ 

L-  4-» 

4-1 

in 

in 

Li 

in  :x 

OJ 

in 

Li 

in 

L. 

L- 

4-3 

10 

u 

c 

c 

O 

H 

(0  4-3 

u 

in 

3 

in 

3 

4-J 

in  x 

QJ 

o 

a 

ai 

10 

C  H 

Li 

OJ 

4-3 

E 

QJ 

4-3 

E 

QJ 

in 

0) 

OJ 

L. 

•H 

■H 

QJ 

•H 

(0 

0 

»H 

u 

0) 

*-i 

u 

QJ 

QJ 

> 

-H  3 

Li 

in 

m 

in 

H 

«— 4 

4-1 

-U 

e  -h 

L- 

Li 

(0 

H 

Li 

to 

in 

f— i 

•H 

X  QJ 

Q 

GJ 

c 

c 

to 

10 

(0 

C 

X 

QJ 

u_ 

X 

QJ 

u_ 

X 

to 

X 

X 

(0  -h 

U 

< — < 

QJ 

QJ 

•H 

*H 

(J 

QJ 

*— i  m 

73 

D 

o 

73 

D 

O 

X 

(0 

u 

•H  > 

X 

e 

6 

-+J 

4-J 

H 

in 

(0 

C 

Li 

10 

C 

L, 

i 

•H 

QJ 

Li  Li 

CO 

•H 

•M 

c 

c 

<N 

□ 

(0  x 

(0 

QJ 

(0 

a 

QJ 

to 

a 

c 

Li 

"“1 

(0  QJ 

in 

•H 

73 

73 

0) 

0J 

T — 

a 

L.  C 

►— 

X 

6 

-J 

E 

10 

o 

3»  4-J 

QJ 

L« 

4-J 

4-1 

OJ  QJ 

> 

Li 

C 

X 

to 

tO 

cn 

a 

o 

/ — V 

>  E 

/ — V 

/ — s 

Q. 

CD  --h 

to 

T— 

r- 

X 

CL 

(0 

X 

O 

(0 

X 

U 

(0 

X 

u 

Cl 

t— 

E 

X  X  X  X 

a  o  a  a 

in  in  i/i  vn 


LJ  s 
CD 

U3  ^ 


(0 

(0 

CL 

CL 

4-J 

X 

0) 

oti 

•4 

' — ^ 

<*} 

c 

to 

co 

(0 

(0 

X 

Li 

3 

rH 

<T 

u 

rO 

U  /— v 

u 

JC  ^ 

CD  ' 

L, 

/ — \  / — \ 

/ — ^ 

t - s 

<H 

/ — \ 

^-1 

1 

□ 

u 

CD 

L3  LO 

X 

in  lo 

LD 

£  *—  ^ 

*—  r 

4“ 

ni 

(Nj 

n* 

m 

m 

X 

^  1  *3 

o 

OC-  ON  O 

O 

P- 

c 

r— 

C 

T— 

3 

e 

CN 

to 

o  to 

~  fo 

l n 

in  ii  tn  ii  m 

ii  in 

11 

c 

II 

C 

II 

C 

10 

II 

II 

4J 

II 

4-J  II 

£  X  E  2  £ 

z  £ 

z 

o 

Z 

a 

z 

QJ 

Z 

>^Z 

c 

4-3  Z 

C  Z 

□  ' — '  Q  ' — *  Q 

^  Q 

w 

LJ 

■. _ y 

LJ 

w 

QJ 

w 

fO  w 

to 

(0  ' 

fO  w 

XXX 

X 

u 

u 

U 

£ 

U 

Li 

u 

»— 

E 

E 

o 

CD 

L3 

LD 

reliabilities  for  assessment  evaluations  and  for  several  assessment 
components  seem  sufficiently  high  to  support  their  further  use." 

Further  analytic  studies  also  contribute  frequently  to  studies 
on  internal  validity.  Howard  (1974:11?)  reports  that  the  number  of 
dimensions  ranged  from  10  to  52  in  the  companies  she  studied.  The 
purpose  of  factor  analysis  is  to  break  a  large  set  of  'ariables  or 
dimensions  into  basic  elements  or  structure.  A  comprehensive  summary 
of  factor  analytic  studies  is  given  by  Huck  ( 1977:275-2"  "1 .  As  Huck 
(1977:276)  points  out,  "A  direct  comparison  of  these  factor  'ludiu' 
is  not  possible  due  to  the  differences  in  variables,  variaole  interpre¬ 
tations,  assessment  techniques,  and  objectives  of  each  program."  In  a 
later  article,  Sackett  and  Hakel  (1979:136)  conclude  that  "A  small 
number  of  dimensions,  namely,  leadership,  organizing  and  planning,  and 
decision  making  serve  to  define  the  underlying  factor  structure." 

A  recent  article  by  Norton,  Gunne,  and  Thornton  (198C)  includes 
a  number  of  techniques  applied  to  an  internal  validity  study.  Among  the 
techniques  used  are  the  following:  correlations  among  final  dimension 
ratings;  correlations  between  final  dimension  ratings  and  overall  assess¬ 
ment;  correlations  of  dimensions  observed  in  simulations  with  final 
dimension  ratings  and  overall  assessments;  inter-rater  reliability; 


factor  analysis;  and  multiple  regression  of  final  dimension  ratings  on 
overall  assessment.  This  article  provides  a  thorough  example  of  an 
internal  validity  study. 

Although  a  great  deal  of  research  has  been  done  to  show  the 
validity  of  the  assessment  center  technique.  the  validity  of  a  particular 
assessment  center  should  not  be  taken  for  granted,  noses  (1975:9) 


5? 


emphasizes  this  point  as  follows: 


The  historical  record  of  validity  .  .  .  cannot  be  taken  as 
a  guarantee  that  a  given  assessment  program  will  or  will  not  be 
valid  in  a  given  setting.  Because  of  this,  each  user  must  ascer¬ 
tain  the  validity  of  the  program  as  applied  to  one's  organization. 


Military  Applications 

The  final  portion  of  this  literature  review  is  concerned  with 
the  use  of  assessment  centers  in  the  military.  A  number  of  research 
reports  have  been  written  concerning  the  use  of  the  assessment  center 
in  military  organizations.  Six  of  these  reports  are  summarized  in  the 
following  paragraphs. 

Among  the  military  services,  the  U.S.  Army  has  conducted 
perhaps  the  most  extensive  research  on  assessment  centers.  Smith  (19?8) 
provides  an  overview  of  the  pilot  assessment  center  established  in  1973 
at  the  U.S.  Army  Infantry  School  (USAIS)  at  Fort  Benning,  Georgia.  The 
initial  Army  research  had  two  major  purposes,  stated  as  follows  by 
Smith  (1978:1): 

One  was  to  determine  the  validity  of  the  assessment  process 
for  predicting  performance  in  OfFicer  Candidate  School  and  also 
in  junior  officer  assignments  ....  The  second  purpose  .  .  . 
was  to  test  the  value  of  assessment  and  counseling  as  a  personal 
career  development  aid. 

The  pilot  program  was  completed  in  December  1974.  Analysis  of  initial 
results  was  limited  to  inter-rater  reliability  and  factor  analysis. 
Plans  were  also  made  to  collect  data  on  future  performance  for  sub¬ 
sequent  validity  studies.  Oetails  on  the  Army  research  are  described 
in  numerous  foilow-on  reports. 

Of  special  interest  to  this  thesis  is  a  report  on  the  USAIS 
assessment  center  written  by  Salter  and  Omstead  (1974).  Their  report 
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focused  on  two  areas.  The  first  area  of  research  involved  development 
of  an  instrument  for  collecting  performance  data  to  be  used  in  validation 
studies.  The  other  area  of  research  was  an  analysis  of  feedback  methods 
for  conducting  post-assessment  counseling  sessions.  All  counselors  in 
the  Army  program  also  served  as  assessors,  however  "a  counselor  may  or 
may  not  have  previously  served  as  an  assessor  for  an  individual  later 
assigned  to  him  for  counseling1'  (Salter  and  Glmstead,  1974:38)  The 
elements  of  a  typical  counseling  session  are  described  in  Figure  4-1. 

As  part  of  the  analysis,  counselors  were  asked  what  methods  seemed 
particularly  effective  in  capturing  the  assessee's  attention  and  moti¬ 
vating  him.  Some  of  these  methods  are  described  in  Figure  4-2. 

Smith  (1977)  examines  the  use  of  the  assessment  center  tech¬ 
nique  at  the  U.S.  Air  Force  Squadron  Officer  Scnool  (SOS),  ne  descrioes 
the  use  of  assessment  centers  at  SOS  during  1975-1577  and  discusses  the 
importance  of  counseling  in  the  SOS  program.  The  primary  emphasis  of 
Smith's  report  is  an  the  use  of  assessment  in  the  leadership  development 
of  SOS  students,  FHe  discusses  how  an  assessment  and  development  program 
can  be  combined  and  provides  an  example  to  illustrate.  Smith  (1977:42) 
"credits  SOS  with  making  a  determined  effort  to  improve  its  leadership 
development  program  but  suggests  there  is  still  room  for  further  growth 
and  improvement."  The  main  area  of  improvement  called  for  is  to  offer 
both  section  commanders  and  students  more  spedific  guidance  about  the 
assessment  process  and  its'  place  in  the  SOS  Leadership  Development 
Program  (Smith,  1977:42). 
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program.  According  to  UlcAdoo  (1979:3),  the  purpose  of  his  study  was  to 
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Phase  I;  Establishing  rapport  -  Casual  conversation  to  establish  a 
relaxed,  informal  climate  and  reduce  possible  anxiety. 

Phase  lli  Introduction 

A.  (Discussion  of  the  concept  and  purpose  of  the  counseling  session. 

1.  Mutual  agreement  on  the  role  of  the  counselor  and  the  role 
of  the  counselee. 

2.  Encouragement  of  active  assessee  participation  and  note 
taking  during  the  session, 

B.  Revieui  and  discussion  of  the  assessment  process. 

1.  Oiscussion  of  exercises  (e.g.,  naming  and  reviewing  the 
actual  exercise,  discussion  of  the  development,  and 
purpose  of  some  or  all). 

2.  General  discussion  of  the  purpose  and  background  of  the 
leadership  dimensions. 

Phase  III.  Feedback  of  assessment  results  for  each  dimension.  The 
following  steps  occur  for  each  dimension; 

A.  Name  dimension  and  ask  assessee  to  define  iu. 

B.  State  the  official  definition  and  reach  consensus  with  the 
assessee. 

C.  Ask  the  assessee  to  analyze  his  performance  on  the  dimension. 

D.  Provide  the  counselor's  findings  concerning  the  assessee 's 
performance. 

E.  Support  the  assessment  conclusions  with  examples  of  specific 
behavior  from  assessor's  reports  and/or  videotape  segments, 
if  appropriate. 

F.  Encourage  two-way  discussion  of  the  feedback. 

G.  Ask  assessee  to  suggest  ways  to  improve  performance  on  this 
dimension,  if  needed. 

Fi,  Present  counselor  recommendations  for  corrective  actions  or 
remedial  programs. 

Phase  IV:  Conclusion  of  counseling  session 

A.  Obtain  general  feedback  from  the  assessee  in  the  form  of 
questions  and  the  assessee's  summary  of  the  counselor's 
observations  and  recommendations. 

B.  Summarize,  administer  post-counseling  questionnaire,  and 
terminate  session  with  an  effort  to  establish  continued  rapport. 


Figure  4-1 

u5A  15  ASSESSfiiENT  CENTER  ; 

ELEMENTS  GF  A  TYPICAL  COUNSELING  SESSION. 
(Salter  and  Glmstead,  1974:42) 
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I.  High-impact  “’edback 

A 4  Verbal  feedback  from  counselor  followed  by  videotape  support. 

B.  Comparison  of  the  assessee's  performance  with  that  of  his  peers. 

C.  Participative  interaction  which  aids  the  assessee  in  analyzing 
own  actions  and  achieving  insight  into  his  own  weaknesses. 

D.  A  counseling  approach  which  convinces  the  assessee  that  the 
counselor  has  a  genuine  concern  and  interest  in  him  as  an 
individual . 

II.  motivating  approaches. 

A.  Asking  assessee  to  suggest  a  course  of  action  he  might  take  to 
improve  his  performance  in  a  particular  area. 

3.  Demonstrating  the  relevance  of  skill  in  a  particular  dimension 
to  present  real-life  army  situation  and  to  future  career 
progression. 

C.  Sharing  of  personal  experiences  in  which  the  counselor  had 
found  skill  in  an  area  to  be  particularly  important. 

II.  Additional  motivating  methods. 

A.  Individualized  instruction  programs  in  the  Infantry  School 
tailored  to  meet  the  needs  of  the  individual  assessee. 

B.  Periodic  follow-up  interview  either  on  a  regular  basis  or 
upGn  the  initiative  of  the  assessee. 


Figure  4-2 

USAIS  AS5E5SmENT  CENTER: 
EFFECTIVE  COUNSELING  TECHNIQUES 
(Salter  and  Olmstead,  1974:43-44) 
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establish  "the  criterion  validity  of  the  dimensions  of  behavior  used 
at  SOS  when  assessing  students. *'  His  study  may  be  divided  into  two 
areas. 

First,  he  computed  the  cot  relations  of  the  assessment  center 
dimension  ratings,  pretest  scores,  and  reading  scores  with  performance 
at  SOS.  These  results  are  shown  in  Table  4-4.  The  academic  score  was 
a  raw  score  based  on  test  results.  The  leadership  evaluation  was  a 
consensus  of  nine  faculty  members  based  on  their  "judgment  of  how 
effective  the  individual  was  in  influencing  a  group  to  accomplish  a 
specific  task"  (McAdoo,  1979i23-24).  The  overall  performance  was  based 
on  both  acadendc  score  and  leadership  evaluation,  plus  writing  and 
speaking  grades.  Note  that  academic  score,  leadership  evaluation,  and 
overall  performance  were  also  included  as  dimensions  in  Table  4-4. 

The  other  part  of  IT!cAdocj's  study  consisted  of  a  regression 
analysis  using  leadership  evaluation  as  the  criterion  variable  and  the 
assessment  center  dimension  ratings  as  the  potential  predictor  variables. 
His  results  indicated  that  acceptance  of  responsibility,  forcefulness, 
and  interpersonal  skills  were  the  strongest  predictors  of  final  leader¬ 
ship  evaluation. 

In  addition  to  reporting  the  results  of  his  analyses,  McAdoo 
made  several  recommendations.  Perhaps  the  most  significant  recommen¬ 
dation  was  that  the  assessment  ratings  of  SOS  students  be  retained  as  a 
basis  for  a  longitudinal  study  of  leadership  in  the  Air  force. 

Francis  (1978)  describes  some  problems  in  applying  traditional 
methods  to  Air  Force  evaluation,  promotion,  and  selection  systems.  She 
discusses  the  use  of  the  assessment  cenr.er  as  a  possiole  solution  to 
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TABLE  4-4 


CORRELATION  RESULTS  QF  McADDO'S  STUDY 


Dimension 

Leadership 

Evaluation 

Overall 

Performance 

Academic 

Score 

Acceptance  of 
Responsibility 

.519 

.482 

.208* 

forcefulness 

.482 

.424 

.365 

Willingness 
to  Lead 

.475 

.382 

.293 

Motivation 

.3C0 

.433 

.213* 

Interpersonal 

Skills 

.372 

.292 

.138* 

F lexibility 

.321 

.205* 

.106* 

Organizing 

Skills 

.311 

.2y9 

.215* 

Problem 

Solving 

.253 

.330 

.235 

Pretest 

.361 

.342 

.627 

Reading  Test 

.268 

.340 

.496 

Leadership 

Evaluation 

1.000 

.687 

.402 

Overall 

Performance 

.687 

1.000 

.675 

Academic  Score 

.402 

.575 

1 .000 

N  =  83  signi 

.ficance:  p  C  .05,  except 

as  noted 

*p  .05 
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some  of  these  problems.  Areas  mentioned  for  application  of  assessment 
techniques  include  initial  officer  selection,  several  applications  for 
development,  and  selection  of  personnel  for  sensitive  jobs.  Francis 
(1978:54)  concludes  as  follows: 

2  Li. 5.  Air  Force  has  a  number  of  personnel  management 
problems  such  as  selection,  evaluation,  promotion,  development, 
and  retention  which  might  profit  from  the  use  of  an  objective 
and  unbiased  technique  such  as  the  assessment  center  process. 

In  a  similar  study  Geiger  (1977)  examined  the  use  of  the  assess¬ 
ment  center  in  screening  applicants  and  selecting  candidates  for  the 
ITlarine  Corps  Officer  Candidate  School  program.  The  following  comments 
were  offered  by  Geiger  (1977:4)  concerning  the  importance  of  the 
selection  and  development  process: 

Since  the  military  is  a  totally  closed  system,  wnereby  all 
assignments  and  all  promotions  are  made  from  uiirnin,  it  is 
essential  that  the  initial  selection  process  and  continuing 
development  of  officers  be  as  discriminating  and  efficient 
as  possible. 

He  investigated  both  conventional  methods  and  the  assessment  method  of 
identifying  leadership  and  management  potential.  Geiger  (1977:ii)  con¬ 
cludes  that  the  assessment  center  "has  potential  as  a  cost  effective 
adjunct  to  present  officer  candidate  selection  methods  and  is  both 
feasible  and  applicable." 

The  review  of  these  six  articles  has  shown  that  the  assessment 
center  process  is  applicable  to  the  military  and  the  Air  Force  in 
particular.  Beyond  merely  discussing  potential  military  applications, 
actual  assessment  centers  conducted  by  both  the  Army  and  Air  Force  were 
referenced. 
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fi  umnar. 

There  is  no  lack  of  literature  regarding  assessment  centers. 

This  review  i-ias  broadly  covered  the  literature  in  describing  the  history 
and  growth  of  assessment  centers,  some  of  the  many  uses,  some  design 
considerations,  the  issue  of  validity,  and  applications  to  the  military. 

Although  a  considerable  amount  of  research  has  been  done, 
additional  research  is  needed  before  assessment  centers  become  operational 
in  the  *ir  force.  The  SQ5  assessment  program  should  be  continued  and 
data  collected  for  lungitudinal  studies,  also,  pilot  assessment  centers 
for  development  and  perhaps  other  purposes  should  be  established  in  other 
settings  to  gather  additional  data  to  je  used  in  establishing  the  effec¬ 
tiveness  of  the  assessment  center  method  in  the  Air  Force.  This  last 
point  is  addressed  later  in  this  thesis. 

with  this  overview,  the  next  cnapter  is  devoted  to  a  detailed 
validity  study  of  a  specific  assessment  center,  which  may  be  useful  as 
a  pilot  assessment  center  for  officer  development. 
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CHAPTER  V 


validation 

ASP  Management  Assessment  Center 

In  1973,  an  assessment  center  was  specially  desicneo  to  evaluate 
Air  Force  civilian  employees  for  first  level  supervisory  positions.  The 
employees  were  scientists  and  engineers  at  iliright-Patterson  AFB  [WPAF5), 
Ohio.  ITiost  were  at  the  GS-12  and  GS-13  level.  This  center  was  operated 
in  19?4  anp  1975  by  the  Aeronautical  Systems  Division  (ASP)  Civilian 
Personnel  Division.  The  assessment  center  was  calleo  the  "ASC  iTanage- 
ment  Assessment  Center"  and  will  be  hereafter  referred  to  as  "the  A5C 
Center." 

At  UJPAF6,  the  joos  performed  t-y  military  scientists  and  engineers 
are  essentially  identical  to  the  jaas  performed  by  their  civilian  counter¬ 
parts.  Because  cf  tnis  and  also  because  the  results  were  used  for  both 
selection  and  development  purposes,  the  ASD  Center  was  examined  for 
future  application  to  management  development  of  Air  force  o'fi:  >rs.  Trio 
examination  included  a  validity  study  wnich  is  reported  in  this  chapter. 

The  validation  effort  was  divided  into  an  examination  o*  content 
validity  and  an  empirical  validity  stuCy .  Since  a  thorough  jee  analysis 
was  performed,  tnu  content  validity  is  briefly  revieweo  arc  sumnanzec. 

1  he  empirical  validity  is  based  on  data  collected  during  tnu  operation 
c  the  ASC  Center,  he  si  .ts  a:*o  conclusions,  i.jsed  on  *.r.e  stat  is,  ileal 


Content  Validity 


As  stated  in  Chapter  IV,  content  validity  is  concerned  with  the 
accuracy  of  the  assessment  center  in  representing  the  target  job.  The 
content  of  the  ASD  Center  is  examined  in  the  following  areas:  job 
analysis,  dimensions,  exercises,  feedback,  assessor  selection,  assessor 
training,  and  trial  run  and  evaluation,  finally,  the  overall  content 
validity  is  summarized. 

Job  Analysis 

The  job  analysis  for  the  A5Q  Center  was  performed  by  Or.  Joseph 
L.  Moses  of  Assessment  Associates,  Inc.  Based  on  interviews  with  six 
second  level  supervisors.  Or.  Moses  prepared  a  Supervisory  Abilities 
Inventory  designed  to  collect  and  rank  characteristics  essential  in  a 
first  level  supervisory  position.  The  Supervisory  Abilities  Inventory 
is  included  as  Appendix  B. 

The  inventory  was  distributed  by  the  A50  Civilian  Personnel 
Office  to  approximately  700  first  and  second  level  supervisors  in  ASG 
and  the  Air  Force  Acionics  Laboratory  (AFAl).  Responses  were  received 
frcm  a  total  of  1*7  supervisors. 

In  addition  to  determining  supervisory  characteristics,  the 
Supervisory  Abilities  Inventory  was  also  used  to  determine  whether  an> 
significant  differences  existed  between  ASC  and  AFAL  concerns  -ig  tneir 
characteristics.  The  final  report  indicated  that  there  were  only  minor 
differences  between  tne  two  organizations  as  to  qualities  needed  in  a 
first  level  supervisor.  The  complete  jot  analysis  is  included  as 
Appendix  L. 
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Dimensions 


The  job  analysis  identified  15  dimensions  critical  to  success 
as  a  first  level  supervisor.  One  dimension,  "originality",  was  dropped 
after  a  trial  run  of  the  assessment  center.  This  was  based  on  the 
recommendations  of  Or.  Moses  and  the  assessors  due  to  problems  in 
observing  tills  dimension  (Bryant,  1980).  The  final  14  dimensions  and 
their  definitions,  as  used  in  the  ASO  Center,  are  given  in  Figure  5*1. 

£  xercises 

The  assessment  center  exercises  were  selected  or  designed  by 
Or.  looses  to  measure  the  dimensions  listed  above.  The  eyercises  used 
in  determination  of  the  overall  assessment  rating  included  a  back¬ 
ground  interview  and  five  simulations.  These  exercises  are  described 
in  Figure  5-2.  Table  5-1  shows  t  °  dimensions  observed  in  each  exercise. 

Feedback 

' hree  types  of  feedbacx  were  given  upon  completion  of  a  center. 
Each  candidate  received  a  personal,  private  interview  with  an  assessor 
a  few  days  after  completing  the  center.  The  interview  lasted  approx¬ 
imately  one  hour  and  included  a  comprehensive  analysis  of  the  candidate's 
performance  during  the  assessment  center.  Prior  to  the  interview,  each 
candidate  was  asked  to  complete  a  "Career  Development  Inventory"  which 
wti^  then  discussed  during  the  interview  and  recommendations  made  wr.ere 
appropriate.  The  candidate's  first  ievel  supervisor  aiso  received  a 
verbal  rtuort  ‘ron.  the  assessor,  and  a  notice  of  the  overall  assessment 
rating.  second  level  supervisor  u»  tne  candidate  received  a  written 

report  prepared  by  t’e  assessor. 
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1.  Oral  Communications  Skills  (OC):  To  what  extent  can  this  individual 
effectively  express  his  ideas  orally'* 

2.  Leadership  [LE):  To  uuhat  extent  is  this  individual  able  to  get 
others  to  work  together  effectively  in  a  group? 

3.  Energy  [EN):  To  uuhat  extent  does  this  individual  maintain  a  high 
level  of  activity? 

4.  Forcefulness  (FQ):  To  uuhat  extent  is  this  individual  able  to  command 
attention  from  others? 

5.  Persuasiveness  [PE):  To  uuhat  extent  is  this  individual  able  to  sell 
nis  point  of  view  to  others? 

6.  Flexibility  (FL):  To  uuhat  extent  can  this  individual  adjust  to  neuu 
situations  easily? 

7.  Empathy  LEM):  To  uuhat  extent  does  this  individual  exhibit  a  concern 
for  others? 

8.  Stress  Tolerance  (ST):  fo  uihat  extent  is  this  individual  able  to 
respond  appropriately  to  stressful  conditions? 

9.  Risk  Taking  (RT):  To  what  extant  is  this  individual  willing  to  take 
either  risk  or  responsibility  to  achieve  objectives? 

10.  Acquiring  Information  (Al):  To  what  extent  is  this  individual  able 
to  obtain  information? 

11.  Organizing  Skills  [05):  To  what  extent  can  this  individual  plan  and 
organize  the  work  of  himself  and  others? 

12.  Problem  Solving  [PS):  To  what  extent  can  this  individual  easily  find 
the  best  solution  to  a  problem? 

13.  Decision  Making  [DM):  To  whac  extent  can  this  individual  make 
decisions  of  high  quality? 

1 4 .  Written  Communications  Skills  (WC):  To  what  extent  can  this  individual 
effectively  express  his  ideas  in  writing? 


Figure  5-1 

ASD  ASSESSMENT  CENTER  DIMENSIONS 
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1.  Interview  (INT]:  This  exercise  provides  an  opportunity  for  the 
assessor  to  gather  inputs  directly  from  the  candidate  in  a  one- 
on-one  situation.  It  addresses  such  areas  as  job  likes  and 
dislikes,  career  planning,  interests,  and  any  other  areas  the 
candidate  wishes  to  discuss.  Prior  to  the  interview,  each  candidate 
is  asked  to  complete  a  questionaire  which  asks  for  information  re¬ 
garding  background,  interests,  likes  and  dislikes. 

2.  City  Council  (CIC):  This  simulation  "is  an  assigned  role,  leader¬ 
less  group  discussion.  Six  participants  roleplay  City  Council 
members,  who  have  been  called  together  to  determine  the  best  use 

to  be  made  of  a  Federal  financial  grant  of  $1,000,000  to  their  city. 
Each  ‘council  member*  has  information  on  the  needs  of  one  City 
Department.  Each  participant  is  to  try  to  get  the  largest  appro¬ 
priation  possible  for  his  department  while  still  helping  the  group 
swiftly  and  faifly  accomplish  its  task"  (Development  Dimensions,  1973). 

3.  management  Problems  (MOP):  This  simulation  is  a  leaderless  group 
discussion  with  no  assigned  roles.  The  six  candidates  are  to  act 
as  a  group  of  consultants  asked  to  give  recommendations  concerning 
an  organization's  problems.  The  group  must  come  up  with  a  written 
recommendation  of  a  suggested  course  of  action  for  each  of  four 
problems.  All  group  members  must  agree  on  and  initial  the  recommen¬ 
dation. 

4.  Supervisory  Task  Force  (STF)s  This  simulation  is  a  leaderless  group 
discussion  with  no  assigned  roles.  The  six  candidates  are  to  act 

as  a  special  task  force  assigned  to  decide  upon  the  attributes  needed 
to  successfully  perform  a  first  level  supervisory  job.  Each  candidate 
is  given  a  period  to  study  the  list  and  rank  the  attributes  (plus  any 
they  want  to  add]  in  their  order  of  importance  to  a  first  level  sup¬ 
ervisor.  The  candidates  must  then  meet  together  and  prepare  a 
written  report  listing  the  agreed-upon  attributes. 

5.  Research  Budget  (REB]:  This  is  an  individual  exercise  in  which  the 
candidate  is  given  a  short  description  of  a  situation  demanding  an 
immediate  decision.  The  candidate  is  asked  to  play  the  role  of  a 
newly  appointed  personnel  director  who  must  make  an  immediate  decision 
and  present  the  reasoning  behind  it.  A  staff  member  is  assigned  to 
play  the  role  of  a  resource  person  who  will  answer  any  questions  the 
candidate  wishes  to  ask. 

6.  Organizational  Problem  (0RP):  This  is  an  individual  exercise  in  which 
the  candidate  is  asked  to  play  the  role  of  a  special  task  force  mem¬ 
ber.  The  task  force  is  concerned  with  improving  organizational 
effectiveness.  The  candidate  is  tpld  of  a  number  of  concerns  being 
raised  which  suggest  that  the  present  form  of  the  organization  is 
outmoded.  The  candidates  must  prepare  a  written  proposal  outlining 
their  thoughts,  ideas,  and  sunnestions  in  this  matter. 


Figure  5-2 

A5D  ASSESSMENT  CENTER  EXERCISES 
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DIMENSION-EXERCISE  MATRIX 
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Decision  Making  (DM 


Assessor  Selection 


Three  assessors  and  a  chairman  were  required  to  assess  each 
group  of  six  candidates.  Assessors  were  selected  from  sic^.n  level 
supervisors  or  experienced  first  level  supervisors  nominated  by  ^he 
organizations  serviced  by  the  ASO  Civilian  Personnel  Office.  Botn 
military  and  civilian  supervisors  were  considered  for  assessor  assign¬ 
ments.  The  following  selection  criteria  were  suggested  by  Or.  Moses: 

1.  Young  (35-40  age  group  if  possible) 

2.  Anility  to  get  away  from  office  problems 

3.  Must  be  perceptive,  sensitive  and  have  empathy  with 
persons  being  evaluated. 

4.  Must  have  ability  to  communicate,  especially  in  writing. 

5.  freedom  from  bigotry 

6.  Second  level  supervisor  (or  higher)  of  key  staff  person 

7.  Should  be  a  person  who  is  respected  by  peers 

It  should  be  noted  that  the  above  criteria  represented  ideal  conditions 
not  all  of  which  were  met  in  every  case. 

Assessor  Training 

All  assessors  attended  a  one-week  training  course  conducted  by 
Or.  Moses.  The  training  included  the  following  general  areas:  back¬ 
ground  on  assessment  centers;  the  assessment  process;  observing  and 
recording  behavior;  recording  dimensions;  giving  feedback;  and  familiar 
ization  with  dimensions  and  exercises.  The  film  "12  Angry  ffien"  and 
videotapes  of  the  various  exercises  were  used  as  training  aids. 

T  r~al  Run  and  Evaluation 

A  trial  run  of  the  Mil,  tenter  was  conducted  curing  lanuar  .  tf'rc 
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June  1974.  A  total  of  82  candidates  were  assessed  by  12  assessors  during 
this  period.  Upon  completion  of  the  trial  run  an  evaluation  was  initi¬ 
ated  to  determine  whether  or  not  to  proceed  with  the  assessment  program. 
As  part  of  the  evaluation,  the  82  employees  who  participated  in 
the  trial  run  were  surveyed  to  gather  feedback  concerning  their  reaction 
and  recommendations.  Overall  the  reactions  were  favorable.  The  results 
of  the  survey  are  summarized  in  Appenaix  D. 

Another  survey  was  designed  to  receive  feedback  from  the  first 
70  managers  (second  level  supervisors  of  the  82  assessees)  who  had 
received  at  least  one  written  report  from  the  ASD  Center.  The  signif¬ 
icant  findings  (Friedman,  1975)  were  as  follows! 

1.  A  significant  percentage  of  the  managers  (94%)  felt  that 
the  information  provided  by  the  Assessment  Center  was  very 
useful  to  the  employee. 

2.  A  majority  of  the  managers  ( 52%)  felt  that  the  information 
was  useful  to  management.  An  additional  35%  felt  it  was  at 
least  somewhat  useful  to  management. 

3.  A  significant  number  f30)  of  personnel  actions  -  some 
favorable  and  some  unfavorable  to  the  employee  concerned  - 
were  taken  based  on  the  Assessment  Center  information. 

4.  Only  twc  of  the  managers  who  responded  did  not  plan  to 
send  other  employees  through  the  Assessment  Center. 

5.  A  majority  of  managers  (70%)  felt  that  trie  Assessment  Center 
report  provided  more  creditable  data  than  the  supervisory 
appraisal  for  merit  promotion  (Supervisory  Evaluation  of 
LmployeePotential  for  first  Level  Supervisory  Positions,  *u  form 
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6.  Five  of  the  managers  recommended  cancellation  of 
the  Assessment  Center. 

?.  A  large  percentage  (46%)  of  the  managers  expressed  concern 
for  the  amount  of  time  required  for  supervisors  to  serve  as 
assessors. 


Summary 

Upon  completion  of  the  evaluation,  a  decision  was  made  to 
implement  the  ASQ  Center  as  an  additional  input  to  the  Merit  Promotion 
System.  Based  an  the  thorough  job  analysis,  assessment  center  design, 
training,  trial  run,  and  evaluation,  the  ASD  Management  Assessment 
Center  appears  to  possess  content  validity. 

Empirical  Validity 

The  empirical  validity  ccnsisted  of  two  parts,  internal  and 
predictive.  The  internal  validity  is  accomplished  (1)  to  determine 
the  reasonableness  and  consistency  with  which  the  information  is  used 
to  determine  the  overall  assessment  and  (2)  to  determine  the  amount  of 
information  that  is  used.  The  predictive  validity  is  accomplished  to 
measure  the  ability  of  the  assessment  center  techniques  to  predict 
managerial  success. 

Internal  Validity 

The  data  used  in  the  internal  validity  consisted  of  the  three 
assessor  ratings  for  each  dimension  observed  in  an  exercise,  within 
the  six  exercises,  47  separate  dimensions  were  assessed.  4t  tne  time 
of  the  consensus  meeting,  a  r'o:  Eucr  dimension  a nd  an  overall 
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assessment  center  rating  was  given  by  each  of  the  three  assessors.  In 
addition,  a  final  consensus  rating  was  also  arrived  at  for  each  of  the 
14  dimansions  and  the  overall  assessment  of  potential. 

The  ratings  for  the  14  dimensions  were  done  on  a  zero  to  five 
scale  while  the  overall  assessment  center  rating  was  given  on  a  one  to 
four  scale.  The  zero  was  used  only  in  the  47  separate  dimension  ratings 
and  indicated  that  behavior  representing  the  dimension  being  evaluated 
was  not  observed.  Table  5-2  shows  the  distribution  of  the  overall 
assessment  ratings. 


TABLE  5-2 


distribution  of  overall 

ASSESSMENT 

RATINGS 

N  % 

4 

IT, ore  than  acceptable 

6 

5.6 

3 

Acceptable 

62 

43.4 

2 

Questionable 

41 

2B.7 

1 

Unacceptable 

32 

22.4 

143 

100.0 

The  distribution  indicates  that  there  was  an  even  distribution  of  good 
and  bad  ratings  within  49%  of  the  candidates  being  rated  acceptable  or 
higher  and  51  percent  being  rated  questionable  or  lower.  The  mean 
value  for  the  overall  assessment  rating  was  2.32  with  a  standard 
deviation  of  .39. 

Table  5-3  presor  ts  the  means  and  the  standozc  dev  . ..irons  cf  me 
47  Dimension  ratings  for  the  exercises,  and  the  14  final  dimension  ratines. 


TABLE  5-3 


MEANS  ANO 

STANDARD  DEVIATIONS  FOR 

OIMENS  ION. 

RATING 

AND  FINAL 

OIMENS ION 

RATING 

EXERCISE 

DIMENSION 

INT 

CIC 

MGP 

STF 

REB 

ORP 

FINAL 

OC 

3.92 

(.84) 

3.67 

(.98) 

2.84 

(1.20) 

3.76 

(.97) 

3.76 
( .828) 

LE 

2.72 

(1.15) 

2.89 

(1.21) 

2.78 

(1.13) 

EN 

3.35 

(1.16) 

3.44 

(1.13) 

3.46 

(1.21) 

3.45 

(1.05) 

FO 

3.84 

(.99) 

3.14 

(1.13) 

3.11 

(1.22) 

3.08 

(1.19) 

3.13 

(1.00) 

PE 

2.79 

(1.08) 

2.78 

(1.21) 

3.24 

(1.12) 

2.86 

(.92) 

FL 

3.10 

(1.20 

2.72 

(1.31) 

2.97 

(1.21) 

2.93 

(1.18) 

2.96 

(.90) 

EM 

3.59 

(1.14) 

3.20 

(1.15) 

3.32 

(1.04) 

3.44 

(.81) 

ST 

3.95 

(1.05) 

3.80 

(1.27) 

3.99 

(.98) 

4.08 
(  -85) 

RT 

3.30 

(1.10) 

2.71 

(1.30) 

2.70 

(1.40) 

3.17 

(1.20) 

3.08 
(  .86) 

A I 

2.96 

(1.29) 

3.17 

(1.06) 

3.11 

(1.16) 

3.13 

(.94) 

OS 

3.20 

(1.09) 

3.05 

(1.26) 

3.44 

(1.04) 

3.26 
( *967 ) 

3.25 

(.86) 

PS 

3.07 

(1.21) 

3.11 

(1.20) 

3.30 

(1.07) 

2.98 

(1.06) 

3.17 

(.89) 

OM 

2.99 

(1.21) 

2.92 

(1.32) 

3.36 

(1.08) 

2.99 

(1.02) 

3.10 

(.89) 

U1C 

A' 

3.51 

(1.06) 

3.36 

(1.02) 

3.35 

(.90) 

ifleans/C Standard  Deviations) 

See  Table  5-1  for  explanation  of  dimensions  and  exercises 
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Overall,  candidates  were  seen  as  rating  low  in  leadership,  persuasive¬ 
ness  and  Flexibility.  Higher  than  average  ratings  were  given  in  oral 
communications,  energy,  empathy  and  stress  tolerance. 

There  are  two  possible  explanations  for  the  low  average  ratings 
in  leadership,  persuasiveness  and  flexibility,  first,  since  the  job 
of  a  scientist  or  engineer  may  not  normally  require  much  need  for  these 
dimensions,  it  seems  reasonable  that  the  assessees  would  rate  lower. 
Secondly,  there  may  be  assessor  bias  in  the  rating  of  these  dimensions. 

The  center  design  may  make  it  difficult  to  rate  highly  all  assessees  on 
these  dimensions. 

The  nigh  average  ratings  ir  both  oral  communications  and  energy 
appear  consistent  with  the  type  of  individual  comprising  the  sample 
population.  The  high  average  ratings  for  both  empathy  ana  stress 
tolerance  may  be  due  to  inflation  by  the  assessors.  For  Doth  dimensions, 
the  assessee  will  rate  high  unless  some  negative  behavior  characteristic 
of  these  dimensions  is  observed  during  the  exercise. 

Correlation  of  Dimensions.  Table  5-4  presents  the  correlations 
among  the  14  Final  dimension  ratings  and  the  overall  assessment  ratings. 
With  the  exception  of  empathy,  written  communications  and  in  some  cases 
stress  tolerance,  the  correlations  are  all  above  .40.  The  highest 
correlations  (.77-. 38)  are  among  three  of  the  five  interpersonal  skills; 
leadership,  energy,  and  forcefulness.  This  high  correlation  is  seen  as 
indicating  the  need  for  energy  and  forcefulness  in  order  to  rise  to  the 
position  of  leaoersnip  within  the  group. 

The  second  highest  grcup  correlations  (.7G-.?4j  exists  anont 
three  ji  the  five  administrative  skills;  decision  making,  erotism  solving, 


CORRELATIONS  AMONG  FINAL  DIMENSION  RATINGS  AND  OVERALL  ASSESSMENT 

(N"144) 


■  garizing.  This  is  also  an  expected  result  based  on  the  inter- 
dep.  Pence  of  the  three  dimensions. 

A  review  of  the  correlations  among  the  14  final  dimension  ratings 
and  the  overall  assessment  rating  indicates  that  persuasiveness  r.81), 
leadership  1..77),  decision  making  (.72)>  problem  solving  (.72),  organ¬ 
izing  skills  (.71)  and  forcefulness  (.70)  had  the  highest  correlations. 
These  results  are  fairly  consistent  with  the  data  of  the  Supervisory 
Aoility  Inventory  (Appendix  C),  in  which  leadership,  decision  making 
and  organizing  skills  were  listed  as  ceing  critically  important  for 
supervisory  success  in  the  target  job.  Persuasiveness,  which  had  the 
highest  corraiation,  was  listed  as  somewhat  important,  and  forcefulness 
was  listed  as  usually  important.  Written  comm:  nications  (.39)  and 
empatny  (.35)  nad  tnp  icuest  correlations. 

The  average  correlations  of  the  dimensions  observed  in  the 
exe.cise  with  the  final  rating  for  that  exercise  ano  also  with  the  over¬ 
all  assessment  rating  are  presented  in  table  5  5.  These  correlations 
indicate  the  strength  of  the  r  :.ationsnip  of  the  dimension  measured  in 
the  exercise  witn  the  (1)  final  rating  _r  that  dimension  ana  (2)  the 
overall  assessment  rating.  All  dimensions  were  observed  m  at  least 
two  exercises. 

A  number  of  the  dimensions,  sucn  as  leadership,  energy,  flex¬ 
ibility,  organizing  skills,  written  communications  and  oral  common icat ion 
displayed  high  correlations  for  all  the  exercisps.  This  indicates  tnat 
tne  assessors  consistently  sec  opportunities  to  ocserve  behavior  char¬ 
acteristic  of  these  at  tensions  in  all  exercises. 
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TABLE  5-5 


CORRELATIONS  OF  DIMENSION  RATI  joS  FOR  EXERCISES  WITH 


FINAL 

RATINGS  ON 

SAME  DIMENSIONS 

AND  WITH 

OVERALL 

ASSESSMENT 

(N=144) 

Dimension 

I  NT 

CIC 

MGP 

5TF 

REB 

ORP 

OC 

.61 

(.41) 

.66 

(.49) 

.67 

(.53) 

LE 

.78 

(.64) 

.77 

(.70) 

.75 

(.59) 

EN 

.75 

(.54) 

.75 

(.58) 

.76 
( .54) 

FO 

.62 

(.A3) 

.74 

(.56) 

.76 

(.60) 

.76 

(.59) 

PE 

.  65 
(.55) 

.72 

(.63) 

.60 

(.55) 

FL 

.63 

(.43) 

.59 

(.55) 

.60 
( .46) 

.59 

(.40) 

cm 

.5? 

(.29) 

.60 

(.26) 

.46 

(.30) 

ST 

.57 

(.25) 

.42 
( -29) 

.63 

R  ' 

.AO 

(.38) 

.  54 
(.36) 

.41 

(.38) 

.44 

(.20)* 

A I 

.38 

(.33) 

.46 

(.50) 

.78 

(.37) 

OS 

.59 

(.59) 

.63 

(.57) 

.67 

(.47) 

.57 
(  .44) 

PS 

.59 

(.59) 

.56 

(.58) 

.66 

(.51) 

.62 

(.44) 

om 

.57 

(.54) 

.47 

(.52) 

.65 

(.50) 

.65 

(.46) 

wc 

.68 

(.31) 

.69 

(.29) 

Final  Ratings/(Ovsrall  Assessment  Rating) 

Significance)  p  £  .01  except  as  noted 
*  .01  <  p  <,  .05 

See  Table  5-1  for  explanation  of  dimensions  and  exercises 
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Acquiring  information  had  a  wide  range  (.38-. 78)  in  correlations 
The  correlation  was  highest  for  the  Research  Budget  and  lowest  for  the 
Interview.  These  results  indicate  that  the  assessors  consistently  saw 
less  opportunity  to  observe  behavior  characteristic  of  that  dimension 
in  the  Interview  than  in  the  Research  Bucket.  These  results  may  be 
reasonable  since  the  Research  Budget  was  designed  to  provide  the  assesse 
with  the  opportunity  to  gather  information  while  Li .e  interview  was  a 
general  information  input  from  the  arsessee  to  the  assessor. 

The  range  for  the  correlations  between  the  dimension  ratings  for 
each  exercise  and  the  overall  assessment  rating  was  .20  to  .70.  Four  of 
the  dimensions,  empathy,  stress  tolerance,  risk  taking  and  written 
communications  had  rather  low  correlations.  Norton,  Dunne,  and  Thornton 
C  1979 s 1 7 )  point  out  that  this  low  correlation  would  arise  if  "either 
the  candidates'  performance  on  tt at  dimension  for  that  simulation  was 
hignly  divergent  from  their  overall  performance  or  that  the  assessors 
were  ignoring  that  particular  dimension  for  that  exercise.”  The 
correlations  for  these  three  dimensions  were  low  for  all  the  exercises 
indicating  that  probably  the  assessors  were  ignoring  that  dimension  in 
determining  the  overall  rating. 

Inter-rater  reliability.  As  a  measure  of  reliability,  inter- 
rater  reliabilities  for  the  47  dimension  ratings  for  the  exercises  and 
for  the  14  final  dimension  ratings  were  determined  (Table  5-6).  As 
stated  in  Chapter  II,  these  reliabilities  were  computed  by  first 
averaging  the  correlations  of  the  three  assessors  for  each  of  the  24 
times  the  assessment  center  was  conducted.  Only  centers  with  data 
available  for  a.ll  six  assessees  and  which  had  computable  correlation 
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TABLE  5-6 

INTER-RATER  RELIABILITIES  OF  0IIY1EN5I0N  RATINGS  FOR  EXERCISES 
AND  FGR  FINAL  DIMENSION  RATINGS 


limension 

INT 

CIC 

MGP 

STF 

RES 

GRP 

MEAN 

F  INAL 

OC 

.62 

C-93) 

.69 

(.87) 

.69 

(.87) 

.67 

(.86) 

.77 

(.91) 

LE 

.82 

(.93) 

.85 

(.94) 

.83 

(.94) 

.83 

(.94) 

.85 

(.94) 

EN 

.81 

(.93) 

.81 

(.93) 

.68 

(.96) 

.83 

(-94) 

.84 

(.94) 

FO 

.73 

(.32) 

.81 

(.93) 

.04 

(.94) 

.79 

(.92) 

.79 

(.92) 

.84 

(.94) 

PE 

.72 

(.89) 

.82 

(.93) 

.79 

(.92) 

*77 

(.91) 

.82 

(.93) 

FL 

.74 

(.90) 

.79 

(.92) 

.72 

(.59) 

.65 

(.35) 

.73 

(.89) 

.77 

(.91) 

Em 

.70 

(•88) 

.67 

(.86) 

.58 

(.81) 

.65 
( .85) 

.77 

(-91) 

ST 

.60 

(.52) 

.55 

(.79) 

.  5? 
(.80) 

.57 

(.30) 

.72 

(.89) 

RT 

.56 

(.81) 

.71 

(.88) 

.71 

(.88) 

.59 

(.61) 

.  65 
(.85) 

.73 

(.89) 

A I 

.72 

(-89) 

.68 

(.86) 

.79 

(.92) 

.73 

(.89) 

.82 

(.93) 

OS 

.72 

(.89) 

.84 

(.94) 

.72 

(.89) 

.56 

(.79) 

.71 

(.88) 

.78 

(.91) 

P5 

.71 

(.88) 

.79 

(.92) 

.73 

(.69) 

.57 

(.80) 

.70 
f  Q,p  ^ 

.80 

(.92) 

om 

.76 

(.90) 

.74 

(.90) 

.66 

(.85) 

.49 

(.74) 

•  OQ 

(  .85) 

•  73 
(.39) 

wo 

.77 
(  -91) 

.52 
(  .76) 

.65 
(  *65) 

.74 

(.90) 

fflean 

.67 

(.86) 

.73 

(.89) 

.76 

(.90) 

.77 

(.91) 

.69 

(.87) 

.53 

(.77) 

Average  r/(Spearman-Brown ) 

5de  Tauic  5-1  ~or  explanation  of  dimensions 
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coefficients  were  used.  The  average  correlations  of  the  24  groups  were 
then  averaged  to  provide  an  overall  average  for  the  4?  dimension  ratings 
for  the  exercises.  The  Spearman-Brown  formula  was  then  applied  to 
these  averages  to  provide  the  reliability  measure. 

The  inter-rater  reliabilities  ranged  from  .74  to  .94  for  the 
47  dimension  ratings  for  the  exercise.  The  lowest  reliability  for  a 
dimension  within  an  exercise  was  decision  making  in  the  Organizational 
Problem.  Since  decision  making  had  high  reliabilities  for  the  other 
exercises,  this  low  correlation  was  seen  as  a  proolem  with  the  exercise 
not  the  dimension.  The  highest  reliability  was  for  energy  in  the 
Supervisory  Task  force  exercise. 

lable  5-S  also  shows  the  mean  reliabilities  for  l-_,,  exercise 
ann  for  each  dimension.  Ti,^  dihiur.sion  means  range  from  .2C  for  stress 
tolerance  to  .94  ft.r  both  leadership  and  energy.  The  lowest  reliability 
for  a  final  dimension  is  also  for  stress  tolerance.  This  result  was 
expected  based  on  a  review  of  assessor  evaluation  worksheets  which 
indicated  same  confusion  in  the  rating  of  this  dimension.  The  highest 
reliability  for  a  final  dimension  was  leadership.  The  reliabilities 
for  all  the  final  dimension  ratings  are  consistently  higher  than  the 
mean  rating  for  the  dimensions.  This  is  expected  since  differences  of 
more  than  one  in  final  ratings  were  resolved  during  the  consensus 
meeting. 

A  review  of  tne  mean  reliabilities  for  the  exercise  shows  a 
range  of  from  .77  for  the  Organizational  Problem  to  .91  for  the 
Supervisory  Task  Force.  The  mean  reliability  of  the  Organizational 


Problem  exercise  was  consideraoly  lower  than  the  other  exercises 
indicating  some  difficulty  may  exist  in  the  measurement  of  dimensions 
within  that  exercise. 

Factor  Analysis.  A  factor  analysis  of  the  14  dimensions  was 
accomplished  to  identify  underlying  factors.  An  analysis  of  the  entire 
population  (l\l=44)  indicates  the  existence  of  three  factors  with  eigen¬ 
values  of  7.37,  1.23,  and  1.07  (Table  5-7).  Factor  I  was  comprised  of 
the  five  interpersonal  skills  -  leadeiship,  energy,  forcefulness, 
persuasiveness  and  flexibility.  The  second  factor  consisted  primarily 
of  skills,  problem  solving,  and  decision  making  and  written  communi¬ 
cations.  Stress  tolerance  contributed  to  a  lesser  degree,  eoth  oral 
communications  and  risk  taking  contributea  to  coth  h actor  I  and  factor 
II.  finally,  empathy  determined  factor  III.  These  results  are  generally 
consistent  with  previous  studies  (Norton,  Dunne  and  Thornton,  1979  and 
Sackett  and  Hakel,  1979). 

In  the  study  by  Norton,  Dunne  and  Thornton,  two  factors  emerged. 
The  strongest  factor  consisted  of  Problem  Analysis,  Judgment,  Planning 
and  Organizing,  and  Written  Communications.  This  result  compares  well 
with  the  FACTOR  II  dimensions  of  the  present  study.  The  factor  analysis 
done  by  5ackett  and  Hakel  resulted  in  three  factors,  the  first  two  of 
which  were  common  among  assessors.  The  first  factor  consisted  of 
Organizing  and  Planning,  Decision  Making  and  Decisiveness.  The  second 
factor  consisted  of  Leadership  and  Behavior  Flexibility.  These  results 
are  comparable  to  FACTOR  II  and  Factor  I  respectively . 
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FACTOR 

ANALYSIS  RESULTS: 

LOADING  AFTER  VARIiMX 

ROTATION 

Dimension 

Factor  1 

Factor  II 

Factor  III 

OC 

.51? 

.514 

.052 

LE 

1  .840 

i 

1 

1 

1 

.335 

.127 

EN 

i 

.893 

1 

1 

1 

1 

1 

.169 

.014 

FQ 

1 

!  .88? 
i 

1 

1 

1 

.235 

.022 

PE 

i 

!  .657 

» 

» 

1 

1 

.482 

.19? 

fl 

i 

i  .  584 

i 

1 

1 

1 

1 

.354 

.403 

Eim 

.068 

.062 

[  .945 

i _ _ 

ST 

.291 

i  1 

S  -582  ! 

.170 

RT 

.492 

.468 

.256 

A I 

.287 

[  .666  [ 
i  i 

.251 

OS 

.463 

i  1 

;  .743  | 

1  • 

.007 

PS 

.301 

i  < 

'  .798  J 

i  t 

.173 

Dim 

.375 

i  i 

J  .722  1 

i  i 

.165 

U1C 

.052 

i  .7&6  i 

.154 

Variance  explained  by  the  three  factors:  69.1% 
See  Table  5-1  for  explanation  of  dimensions. 


Since  a  specific  pait  of  the  center  design  was  a  Feedback  Report 
which  was  used  by  assessors  to  pro'/tde  feedback  to  the  assessee  in  four 
areas,  a  factor  analysis  was  accomplished  forcing  in  a  fourth  factor. 

(Table  5-8).  The  results  show  that  Factor  I  is  determined  Dy  the 
interpersonal  skills,  Factor  II  by  the  aoministrative  skills.  Factor  III 
by  the  communicative  skills  and  stress  tolerance,  and  Factor  I'J  by  empathy. 
With  the  exception  of  stress  tolerance,  these  four  factors  are  identical 
to  the  four  areas  on  the  Feedback  Reports. 

In  an  attempt  to  determine  the  extent  to  which  assessors  differ¬ 
entiate  among  the  dimensions,  a  separate  factor  analysis  was  accomplished 
for  each  of  the  24  times  the  center  was  run.  These  24  runs  represented 
a  different  set  of  three  assessors  for  a  set  of  six  assessees.  Cf  these 
24,  one  was  excluded  in  the  analysis  since  both  forcefulness  and  persua¬ 
siveness  resulted  in  zero  factor  coefficients. 

For  the  remaining  runs  the  number  of  factors  ranged  from  2  to  4. 
There  were  three  cases  in  which  two  factors  were  indicated;  ten  cases 
in  which  three  factors  ere  indicated;  and  ten  cases  in  which  four  factors 
were  indicated.  These  results  suggest  that  differences  do  exist  in  the 
nature  of  the  underlying  factor  structure  for  the  different  assessor 
teams.  Some  of  the  more  consistent  findings  were  (1)  Leadership  and 
energy  were  strong  determinants  of  one  of  the  factors  in  20  of  the  23 
runs,  (2)  empathy  strongly  determined  a  factor  in  seven  of  tne  runs,  an  d 
(3)  organizing  skills  and  problem  solving  load  as  one  factor  in  13  of 
the  runs. 

Multiple  Repression  Analysis.  A  ten  step  regression  analysis  was 
accomplished  using  the  14  final  dimension  ratings  to  predict  the  overall 
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TABLE  5-8 


FACTOR  ANALYSIS  RESULTS!  LOAQING  AFTER  UARIIY1AX  ROTATION  (4  FACTORS) 


Oimension 

Factor 

I 

Factor  II 

Factor  III 

Factor  IV 

OC 

.495 

.186 

.658 

.  102 

LE 

J  .805 

1 

1 

1 

.397 

.156 

.088 

EN 

J  .876 

1 

1 

1 

.208 

.158 

.007 

FC 

J  .866 

1 

1 

r 

.244 

.185 

.001 

PE 

j  .615 

i 

1 

i 

.450 

.309 

.166 

FL 

j  .  549 

i 

i 

i 

.  __j 

.388 

.193 

.373 

Bill 

.061 

.115 

.055 

j  .952 

i  __ _ _ 

ST 

.268 

• 

_ * 

CD 

CD 

1  1 

j  .747  ; 

1  -J 

.231 

RT 

.434 

|  .638  j 

.037 

.160 

A I 

.215 

|  . 756  J 

.177 

.149 

OS 

.396 

j  .688  | 

I  1 

.383 

.076 

PS 

.255 

<  1 

j  .785  | 

i  i 

.345 

.  054 

om 

.302 

j  .766  1 

1  1 

.256 

.070 

UIC 

.066 

.402 

1  «  -  ■  ■■  "| 

j  .710  j 

.  144 

I  I 


Variance  explained  by  the  Four  factors:  75.1% 
5ee  Table  5-1  for  explanation  of  dimensions 
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assessment  center  rating  [Table  5-9).  This  procedure  was  done  tc  nuuel 
the  amount  of  information  that  could  be  used  in  making  the  overall 
assessment  rating. 

As  in  previous  studies  (Norton,  Dunne,  and  Thornton,  1579;  and 

Sackett  orb  hakel,  1979),  a  smaller  subset  of  the  total  numder  of 

2 

dimensions  yielded  a  high  R  ,  indicating  that  the  overall  rating  can 
be  modeled  quite  accurately  using  a  portion  of  the  total  number  of 
aimensicns  measured.  In  the  study  by  Morton,  Dunne  and  Thornton,  four 
dimensions  (Planning  and  organizing,  Problem  Analysis,  Oral  communications 
and  Task  Responsibility)  accounted  for  32.2  percent  of  the  variation  of 
tr.e  overall  rating. 

2 

.n  tne  study  by  Sackett  and  hakel,  tnree  predictors  yielded  A  ’s 

ranging  from  .33  tc  ,86  for  the  cifferent  assessor  teams  participating 

in  the  center.  In  no  case  dia  equations  with  up  to  seven  predictors 
2 

exceed  the  R  for  three  predictors  by  more  than  .05.  ’re  three 
strongest  predictors  in  their  study  were  Leadership,  Planning  and  Organ¬ 
izing,  and  Decision  Making. 

TABLE  5-9 

MULTIPLE  REGRESSION  GF  FINAL  □  IITiENS  ION  RATINGS  C'N  OVERALL  ASSESSMENT 


Variable 

R2 

2 

R  Increase 

Results  of  5-step 
3  iJeicht  F 

solution 

Big  of  r 

Pl 

.662 

— 

.341 

41.93 

•  ODD 

LE 

.  745 

.083 

.234 

31.23 

.000 

CM 

.799 

.053 

.250 

27.84 

.ooc 

EM 

.814 

.016 

.135 

10.58 

.001 

RT 

.823 

.009 

.131 

7.27 

.008 

Constant 

-.955 

The  regression  for  the  entire  population  ([\i=l44;  for  the  ASO 
Center  resulted  in  Persuasiveness,  Leadership,  Decision  Waking,  Empathy 
and  Risk  Taking  accounting  for  82.3  percent  of  the  variation  of  the  over¬ 
all  rating.  While  Persuasiveness,  Leadership,  Decision  Waking  and  Risk 
Taking  had  high  correlations  with  the  overall  rating  (Table  5-4), 
empathy  had  the  lowest. 

Predictive  Validity 

The  predictive  validity  consisted  of  a  computation  of  the 
correlation  coefficients  of  the  14  final  dimension  ratings  and  the  over¬ 
all  assessment  center  rating  with  four  variables  chosen  to  be  indicators 
of  managerial  success. 

The  four  variables  used  as  actual  criteria  of  success  were 
present  grade  (GRACE),  present  salary  (SALARY),  most  recent  merit 
appraisal  rating  (APP),  and  the  number  of  promotions  (FROMGTE)  since 
the  assessment  center.  A  literature  review  accomplished  by  Klimoski 
and  Strickland  (1977:380)  identified  these  as  criteria  used  in  previous 
studies.  There  is  by  no  means  strong  support  for  their  exclusive  use. 

They  were  used  in  this  study  because  of  availability  and  as  a  means 
to  compare  the  predictive  validity  of  the  ASC  Center  with  previous 
studies.  Two  possible  criteria  to  be  further  investigated  are  differences 
in  salary  and  the  results  of  a  personal  interview  with  the  assessee's 
immediate  supervisor  on  present  job  performance. 

The  number  of  promotions  for  the  143  candidates  ranged  from 
zero  to  two.  Dnly  4.2  percent  had  two,  32.5  percent  had  one  and  52.9 
percent  had  zerc  promotions  since  che  assessment  center.  Present  grade 
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ranger1  from  GS-11  to  GS-15  with  G5-13  and  65-14  comprising  almost  92  per¬ 
cent  of  the  candidates.  Salary  for  the  candidates  was  dependent  on 
both  G5  rating  and  step  within  the  G5  rating.  The  range  was  from 
$22,671  to  $43,126  with  a  mean  of  $36,674.  There  were  SO  steps  in 
the  salary  range  with  the  most  frequent  occurring  in  20.3  percent  of  the 
cases.  The  range  for  the  most  recent  merit  appraisal  rating  was  from 
40  to  70  with  a  mean  of  65  and  a  standard  deviation  of  5.4.  Over  64 
percent  of  the  110  candidates  having  available  merit  appraisal  ratings 
has  a  rating  of  66  or  higher,  indicating  some  inflation  in  the  appraisal 
system. 

The  correlations  of  the  overall  assessment  center  rating  and  the 
four  criterion  variables  ranged  from  .15  for  salary  to  .28  for  the  most 
recent  merit  appraisal  rating  (Table  5-10/.  The  number  cf  promotions 
displayed  tne  second  highest  correlation  (.27),  just  slightly  lower  than 
that  for  the  appraisal  rating. 

The  predictive  validity  for  the  appraisal  rating  compares  well 
with  the  .30  correlation  of  the  study  by  Norton,  C-unre  and  Thornton 
(1979).  The  predictive  validity  for  the  number  of  promotions  was  some¬ 
what  lower  than  the  .44  obtained  in  the  AT&T  studies  (Howard,  1974L122). 

One  factor  to  consider  with  the  low  correlation  for  salary  is  the  irregular¬ 
ities  in  the  pay  scales.  For  instance,  it  is  not  uncommon  for  a  junior 
G5-14  to  have  a  lower  salary  than  a  senior  GS-13. 

The  correlations  of  the  14  final  dimension  ratings  ana  the  over¬ 
all  assessment  center  are  also  shown  in  Table  5-10.  Excluding  salary, 
cne  corre lat j-ui is  across  the  three  other  criteria  variables  are  fairly 
consistent.  Oral  communications,  leadership  and  written  communications 
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TABLE  5- IQ 


CORRELATIONS  OF  FINAL  DIMENSION  AND  OVERALL  ASSESSMENT  RATINGS 
WiTH  INDICATORS  OF  MANAGERIAL  SUCCE5S  (N=143  except  as  noted) 


Dimensions 

Promotions 

Salary 

APP*** 

Grade 

Mean 

OC 

.18* 

.26 

.31 

.32 

.27 

LE 

.25 

.  18* 

.22 

.25 

.23 

EM 

.23 

.09** 

.19* 

.18* 

.17 

FO 

.27 

.13** 

.20* 

.22 

.21 

PE 

.23 

.09** 

.20* 

.22 

.IS 

FL 

.19* 

-.01** 

.11** 

.  12** 

.10 

EM 

.00** 

-.10** 

.07** 

-.03** 

-.02 

ST 

.20 

.15* 

.27 

.24 

.22 

RT 

.20 

.02** 

.28 

.14' 

.lb 

AI 

.19* 

.07** 

.11** 

.15* 

.13 

GS 

.17* 

.  15* 

.22 

.22 

.19 

PS 

.25 

.11** 

.24 

.23 

.21 

OM 

.29 

.09** 

.24 

.21 

.21 

wc 

.19* 

.18* 

.29 

.25 

.23 

Mean 

.20 

.10” 

.21 

.20 

Overall 

Assessment 

.27 

.15* 

.28 

.24 

***  fj  = 

110 

Significance:  p  < 

•01  except 

as  noted 

*  .01 

< 

P  <  -05 

**  p 

> 

.05 

See  Table  5-1  for  explanation  of  dimensions 
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have  the  highest  m-an  correlation  with  the  four  criteria  variatles. 

Empath  has  by  far  the  lowest. 

Conclusions 

In  general,  the  results  of  the  empirical  validity  shew  that  the 
assessment  center  provides  a  useful  measure  of  the  dimensions  evaluated. 
Some  specific  conclusions  follow: 

1.  Acquiring  information  should  be  excluded  from  the  interview. 
The  correlation  of  this  dimension  with  the  final  dimension 
rating  indicates  that  very  little  weight  was  given  to  this 
dimension  by  the  assessors  in  that  exercise. 

2.  The  organizational  problem  exercise  needs  to  be  examined  for 
possible  problems  in  the  measurement,  of  dimensions  within 
that  exercise.  Inter-racer  reliabilities  for  all  four 
dimensions  within  that  exercise  were  considerably  lower 
than  for  any  other  exercise. 

3.  Definition  problems  exist  for  the  stress  tolerance  dimension. 
This  was  indicated  first  by  the  low  inter-rater  reliability 
and  second  by  reviewing  the  assessor  worksheets.  In  seme 
cases,  the  same  assessee  was  given  both  a  zero  and  a  five 
rating  by  different  assessors  because  no  stress  was  observed. 

A.  Evidence  indicates  that  empathy  Is  a  dimension  independent 
of  all  others  and  has  little  bearing  on  performance  within 
the  assessment  center.  It  displays  little  correlation  with 
any  of  the  other  13  dimensions,  has  the  lowest  correlation 
with  the  overall  assessment  rating  and  consistently  loaos 
as  a  separate  underlying  factor.  This  dimension  was  net 
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originally  identified  on  the  Supervisory  Abilities  Inventory, 
but  was  added  based  on  additional  comments  provided  by 
supervisors  completing  the  inventory.  It  is  t'elt  that 
empathy  can  be  eliminated  from  tne  dimensions  evaluated. 

Summary 

Although  the  assessment  center  design  appears  to  have  content 
validity,  the  empirical  analyses  indicates  the  existence  of  some  pro- 
clems  in  the  usefulness  of  specific  exercises  and  the  measurement  and 
usefulness  of  certain  dimensions.  Internal  validity  and  the  predictive 
validity  of  the  appraisal  rating  are  consistent  with  the  results  found 
in  previous  studies. 

The  following  chapter  provides  the  background  for  conducting 
a  pilot  assessment  center  for  management  development  of  Air  Force 
officers.  A  training  approach  is  described  which  is  aimed  at  addressing 
some  of  the  problem  areas  mentioned  above. 
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CHAPTER  VI 


PILOT  ASSESSMENT  CENTER 

I ntroduction 

Ons  of  the  objectives  of  this  research  effort  is  to  provide  a 
basis  for  a  pilot  assessment  center  to  further  investign  o  the  assess¬ 
ment  center  as  a  means  for  management  development  of  Aii  farce  officers. 

In  Chapter  III,  the  management  development  portions  of  officer  develop¬ 
ment  programs  were  reviewed.  This  review  indicated  a  need  for  a  more 
effective  way  of  providing  feedback  to  officers  concerning  their  manage¬ 
ment  skills.  Criteria  common  to  the  management  development  programs 
were  also  listed.  These  criteria  included  oral  communication  skills, 
written  communication  skills,  problem  solving,  leadership,  decision 
making,  and  resource  management. 

Of  the  above  mentioned  developmental  criteria,  all  except 
resource  management  were  included  as  dimensions  in  the  A5D  fflsnagement 
Assessment  Center.  In  addition,  the  A5D  center  included  the  following 
nine  dimensions!  energy,  forcefolness,  persuasiveness,  flexibility, 
empathy,  stress  tolerance,  risk  taking,  acquiring  information,  and 
organizing  skills.  The  dimensions  used  in  the  ASD  Center  were  the 
result  of  an  indepth  job  analysis  of  first  lovel  supervisory  jobs  in 
the  scientific  and  engineering  field. 

''though  civilian  personnel  were  tie  subjects  of  the  job  analysis, 
it  is  assumed  that  ^he  same  dimensions  would  apply  to  military  officers 
in  similar  jobs.  This  assumption  is  thought  to  be  valid  since  the  jobs 
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performed  by  military  scientists  and  engineers  at  UJright-Patterson  ATB 
are  essentially  identical  to  the  jabs  performed  by  their  civilian 
counterparts.  Because  of  the  similarity  between  military  and  civilian 
jobs,  plus  the  similarity  between  developmental  criteria  and  the  ASO 
Center  dimensions,  the  ASO  Center  was  examined  for  future  use  as  a 
management  development  tool  for  Air  Force  officers. 

The  data  presented  in  Chapter  V  provide  general  support  for  the 
content  and  empirical  validity  of  the  ASO  Center.  Some  of  the  diffi¬ 
culties  with  certain  exercises  and  dimensions  identified  in  that  chapter 
may  be  addressed  by  improvements  in  assessor  training.  Although  addi¬ 
tional  changes  to  the  assessment  center  design  itself  may  be  desirable, 
it  is  recommended  that  the  same  design  be  used  in  the  pilot  center  so 
that  results  may  be  compared  to  results  of  the  1974-1975  ASO  Center. 

This  chapter  provides  the  basis  for  updating  the  assessor  training, 
providing  the  required  assessor  feedback,  and  gathering  the  information 
necessary  to  examine  the  ASO  Center’s  effectiveness  as  a  management 
development  method. 


Assessor  Training 

As  was  painted  out  in  Chapter  IV,  assessors  are  normally  pro¬ 
fessional  managers  two  or  three  levels  above  the  candidates.  While  they 
may  have  an  extensive  background  in  the  management  field,  they  must  also 
receive  adequate  training  to  enable  them  to  accurately  understand  and 
evaluate  the  dimensions  measured  in  the  aseessment  center.  Standards 
and  ethical  considers* inns  for  assessment  center  operations,  Appendix  A, 
presents  the  minimum  training  requirements  and  a  brief  description  of 
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what  should  be  included.  Table  6-1  presents  a  suggested  schedule  for 
assessor  training. 

TABLE  6-1 
ASSESSOR  TRAINING 

Introduction  to  Assessment  Center 
Overview  of  the  Assessment  Process 
Observing  and  Recording  Candidate  Behavior 

Review  of  Assessment  Dimensions 
Review  of  Exer  ses 

Evaluation  of  Candidate  Behavior 
Review  of  Benchmarks 
Team  Assessment 

Practice  Assessment  Center 

Writing  Final  Reports 
Feedback  Training 
Assessment  Center  Administration 

Since  measurement  of  the  dimensions  is  a  prime  consideration,  a 
thorough  understanding  of  the  basic  dimension  definitions  and  the 
relevant  behavior  associated  with  each  dimension  should  be  stressed. 
During  the  assessor  training  this  is  accomplished  first  by  acquiring 
an  understanding  of  the  dimtnsion  definitions,  their  relationship  to 
other  dimensions,  and  examples  of  effective  and  ineffective  performance. 
This  understanding  is  further  reinforced  during  the  review  of  the 
specially  developed  benchmarks  which  represent  the  typical  benavior 
expected  for  a  specific  dimension  rating.  Finally,  the  operation  of 
the  practice  assessment  center  and  the  writing  of  final  reports  should 
be  used  to  resolve  any  remaining  confusion. 


Day  One 

Day  Two 
Day  Three 

Day  Four 
Day  Five 
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In  the  ASC  Center,  brief  dimension  definitions  were  printed  an 
the  exercise  report  forms  used  by  the  assessors.  Both  a  review  of  these 
reports  and  the  results  of  the  internal  validation  indicate  that  the 
different  assessors  viewed  some  dimensions  differently.  To  increase 
assessor  understanding  relative  to  the  dimensions,  expanded  dimension 
definitions  and  benchmarks  have  been  developed  for  use  in  the  assessor 
training  (Fig.  6-1  to  6-16). 

The  expanded  definitions  and  benchmarks  are  based  on  -  review  of 
the  assessor  reports  and  worksheets  of  over  20  assessees.  The  review 
was  accomplished  to  determine  consistent  observation  points  across 
different  assessors  for  the  same  dimensions.  In  some  cases,  the  defini¬ 
tions  were  actually  reworded,  however,  in  ail  cases,  additional  points 
to  consider  in  assessing  a  dimension  were  included.  In  the  ASQ  Center, 
dimensions  were  rated  on  a  scale  of  one  to  five.  A  rating  of  one  indi¬ 
cated  very  little  of  a  dimension  was  shown.  A  rating  of  five  represented 
excellent  behavior  far  a  given  dimension.  Benchmarks  were  developed  for 
dimension  ratings  of  two,  three,  and  four. 

Assessor  Feedback 

No  matter  how  valid  the  assessment  center  approach,  it  will  have 
little  value  as  a  management  development  tool  if  the  feedback  to  the 
assessee  is  ineffective.  In  an  assessment  center  used  for  management 
development,  the  primary  objectives  of  feedback  are  learning  and 
motivation.  Salter  and  Qlmstead  (1974:35)  state  that  feedback  should 
result  in  the  assessee's  (l)  learning  the  precise  nature  of  any  undesire- 
able  behavior  produced  or  of  desired  behavior  not  produced,  (2j  learning 
ways  of  correcting  deficiencies,  and  (3)  acquiring  motivation  to  improve. 


-  -  v  r  e*  T  i  i)  fT 63ES I  I  11 
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Oral  Communication  Skills:  How  effective  is  this  individual  in 
expressing  ideas  orally?  Is  presentation  logical  and  well-organized? 
Look  to  the  audience  for  cues.  What  is  their  reaction? 

Additional  points  to  consider: 

Volume  and  projection 

Voice  clarity  and  articulation 

Voice  tone  -  pleasant  and  enthusiastic  or  monotone? 

Eye  contact,  gestures,  and  facial  expressions 
Use  of  notes 
Distracting  habits 
Organization 

Note:  Vocabulary  and  formal  correctness  are  not  critical  unless  they 
distract  from  the  message  being  communicated. 

Benchmark  Two 

Largely  ineffective  in  both  group  and  one-on-one  situations. 

Low  monotone  voice  which  is  barely  audible  at  times.  Looked  down  too 
much  wnen  referring  to  notes  and  had  a  tendency  to  read  from  them. 
Questioning  during  Research  Budget  was  rambling  with  no  main  points. 
Reluctant  to  enter  discussion  during  City  Council. 

Benchmark  Three 

Effective  speaker  in  both  group  and  one-on-one  situation. 

Sppke  clearly  with  goad  volume  and  adequate  variations  in  voice  inflec¬ 
tions.  During  the  City  Council  used  a  number  of  pauses  and  "ok’s"  which 
detract  from  overall  presentation.  Used  good  hand  gestures  and  eye 
contact  was  good  during  Interview  and  Research  Budget,  but  only  fair 
during  City  Council. 

Benchmark  Four 

Very  effective  speaker.  Voice  is  clear,  projects  well  and 
pleasant  to  listen  to.  Good  vacabulary.  Always  maintains  good  eye 
contact  and  makes  good  use  of  notes.  Questioning  in  Research  Budget 
was  concise,  clear,  and  to  the  point.  The  only  deficiency  noted  was 
formal  presentation  in  City  Council  could  have  been  better  organized. 
Could  excell  with  practic  and  polish. 


Figure  6-1 

DEFINITION  AND  BENCHMARKS :  ORAL  COMMUNICATION  SKILLS 


L eader ship !  To  what  extent  is  this  individual  able  to  get  others  to 
work  together  as  a  group  in  accomplishing  a  task? 

Additional  points  to  consider: 

Attempts  to  lead.  It  is  better  to  attempt  to  lead  and 
fail  than  to  never  even  attempt  to  lead. 

Effectiveness  in  coping  with  dissension  and  bringing 
about  compromises. 

Appropriateness  of  leadership  style 
Success  in  gaining  group  support. 

Note:  Remember  that  in  an  assessment  center  it  is  possible  for  all 
candidates  to  receive  the  same  rating.  All  candidates  may  be  rated 
low  or  all  candidates  may  be  rated  high.  See  figure  6-15  for 
relationship  between  Leadership,  Persuasiveness,  Forcefulness,  and 
Energy. 


Benchmark  Two 


Made  few  attempts  to  lead.  These  few  attempts  at  leadership 
were  ineffective.  In  the  City  Council  was  more  interested  in  own 
proposal.  Eager  to  get  discussion  started  but  did  not  sustain,  No 
evidence  of  desire  to  lead  in  other  group  exercises.  Outwardly 
passive  and  content  to  follow  others. 


Benchmark  Three 


Effective  in  a  limited  number  of  attempts  to  lead.  Assumed 
leadership  role  in  the  Supervisory  Task  Force  and  lead  the  group  to  a 
consensus  decision.  Did  not  dominate  the  management  Problems  Exercise 
but  did  impact  group  at  times  by  summarizing.  Was  willing  to  let 
others  take  the  lead  in  City  Council,  but  assisted  in  negotiating  the 
final  compromise. 


Benchmark  Four 


Demonstrated  definite  qualities  of  leadership.  Although  not 
completely  dominating  the  group,  constantly  channeled  the  group  dis¬ 
cussion  and  initiated  its  decisions,  made  a  strong  bid  for  leadership 
of  the  Supervisory  Task  Force  by  stepping  to  the  chalkboard  and  writing 
down  own  list  -  was  able  to  maintain  control.  Contributed  strongly  to 
final  recommendations  on  the  management  Problems  by  asking  pertinent 
questions  and  restating  others'  positions.  During  the  City  Council  dis¬ 
cussion  was  listened  to  attentively  by  the  group  and  rereived  support 
most  of  the  time. 


Figure  6-2 

DEFINITION  AND  BENCHMARKS :  LEADERSHIP 
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Energy t  To  what  extent  does  this  individual  maintain  a  high  level  of 
activity?  How  much  ooes  this  individual  participate  in  exercises? 

Additional  points  to  consider: 

Verbal  inputs 

Nate-taking 

Use  of  chalkboard 

Remember  to  look  for  level  of  activity.  Do  not  be  concerned 
with  the  impact  of  the  activity  on  the  group. 

Ncte:  See  Figure  6-15  for  relationship  between  Leadership,  Persua¬ 
siveness,  Forcefulness,  and  Energy. 


Benchmark  Two 


Very  low  level  of  energy.  Almost  a  non-participant.  After 
initial  presentation  participated  very  little  in  City  Council  Exercise, 
responding  only  when  questioned.  In  other  exercises,  participated  very 
little,  making  only  sporadic  comments. 


Benchmark  Three 


Shows  greatest  level  during  planning  and  organizing  phases. 
Great  discussion  starter  but  fades  rapidly.  Took  many  notes  during 
City  Council.  Paricipated  in  a  lively  fashion  during  early  discussion 
period  but  did  not  sustain.  In  the  Management  Problems  Exercise, 
worked  at  a  steady,  but  not  vigorous  pace  during  the  planning  period. 
Participated  in  the  discussion  but  never  very  actively.  In  the 
Supervisory  Task  Force  summarized  occasionally  and  volunteered  to 
write  final  decision. 


Benchmark  Four 


Displayed  a  great  deal  of  energy.  Entered  the  discussion  almost 
immediately  and  participated  almost  constantly  thereafter.  Interested 
and  enthusiastic  throughout  discussion.  Was  ouick  to  volunteer  to  help 
write  up  the  recommendation  to  the  Management  Problems.  Set  the  pace 
for  the  City  Council  Exercises,  making  frequent  inputs  and  asking 
questions.  Worked  at  chalkboard  to  keep  track  of  rankings  on  Super¬ 
visory  Task  Force  and  maintained  a  steady  input  of  iueas. 


Figure  6-3 

DEFINITION  AND  BENCHMARKS:  ENERGY 
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Fcrcefulness:  To  what  extent  is  this  individual  able  to  command  attention 

from  others?  Is  this  individual's  presence  felt  by  others?  Is  there  any 
attempt  to  actively  seek  attention  rather  than  passively  react  to  events 
as  they  occur? 

Additional  points  to  consider: 

Aggressiveness 

Audience  attention  and  reaction 

Self-confidence 

Oisplay  of  initiative 

Note:  The  difference  between  leadership  and  forcefulness  is  that  while 

an  individual  may  be  forceful,  i.e.,  able  to  command  attention,  this 
individual  may  not  be  effective  in  leading  others  to  accomplish  a  task. 
Also,  forcefulness  is  concerned  with  the  effect  of  an  individual's 
activity,  while  energy  is  concerned  only  with  the  amount  of  activity. 

The  relationship  between  leadership,  persuasiveness,  forcefulness  and 
energy  is  shown  in  figure  6-15. 

Benchmark  Two 


Quiet  and  reserved.  Commands  attention  on  only  a  short-term 
basis.  Manner  of'  speaking  is  generally  low-keyed.  During  City  Council 
was  quite  willing  to  change  positions  to  accomodate  challenges,  most 
of  inputs  were  ignored  by  others  in  the  Supervisory  Task  Force.  Some¬ 
what  passive  during  the  discussion  of  the  management  Problems  -  willing 
to  "go  with  the  flow."  Seemed  unsure  of  self  during  the  interview. 


Benchmark  Three 


Able  to  hold  attention  of  others  but  tends  to  hold  back.  Came 
on  strong  in  the  Interview,  but  not  as  aggressive  in  group  discussions. 
City  Council  presentation  was  Forceful  and  listened  to  by  the  otners. 
Aggressive  in  support  of  his  department,  but  a  bit  reluctant  to  inter¬ 
ject  self  too  strongly  into  the  discussion.  During  discussion  of  manage¬ 
ment  Problems  and  on  the  Supervisory  Task  Force,  others  reacted  favorably 
to  the  few  inputs  made  by  #3. 

Benchmark  Four 


Displayed  considerable  forcefulness  throughout  the  discussions 
stating  positions  and  posing  questions  and  alternatives  in  a  knowledgeable 
and  confident  manner.  When  this  individual  made  comments  on  the  manage¬ 
ment  Problems,  ethers  listened  attentively  and  showed  respect,  since 
many  of  the  comments  were  accepted.  Was  very  effective  in  focusing  the 
group's  attention  an  their  department  in  the  City  Council.  Was  the  cen¬ 
ter  o f  attention  -in  tho  Supervisory  Tack  Force.  V/erv  annrpssivp  in  the 
interview . 

Figure  6-4 

DEFINITION  ANO  BENCHmARKS:  FORCE  FULNESS 
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Persuasiveness;  To  uihat  extent  is  this  individual  able  to  sell  his/her 
point  of  view  and  counter  abjr-tions  from  others? 

Additional  points  to  consider: 

Attempts  to  sell  ioeas.  The  fact  that  a  person  trys  is  worth 
more  than  if  they  did  not  even  try. 

Success  in  selling  ideas 

Was  this  individual  convincing? 

Note:  The  difference  between  Leadership  and  Persuasiveness  is  that  an 
individual  may  be  able  to  sell  ideas  to  others,  but  ineffective  in 
bringing  others  together  to  accomplish  the  whole  task.  See  ciQ"r'e 
6-15  for  the  relationship  between  Leadership,  Persuasiveness,  force- 
fulness  and  Energy. 


Benchmark  Two 


few  attempts  to  sell  ideas  were  largely  ineffective.  Not  very 
convincing  in  City  Council  -  other  group  members  persuaded  #2  more  than 
#2  persuaded  them.  Was  difficult  to  follow  ideas  presented  on  Manage¬ 
ment  Problems  -  did  not  make  specific  points.  Not  at  all  persistent 
in  asking  questions  on  Research  Budget. 

Benchmark  Three 


Makes  points  well  in  some  situations.  More  convincing  in 
initial  City  Council  Presentations  but  later  attempts  to  support  own 
department  were  moderately  successful.  Arguments  and  ideas  were 
generally  good  concerning  the  Management  Problems  and  many  were 
accepted.  However,  the  force  of  arguments  were  carried  by  others. 
Swayed  the  group  away  from  preselection  of  trainees  on  Problem  #1. 

Was  moderately  successful  at  selling  initial  decision  on  Research 
8udget.  When  new  data  was  presented,  was  not  persuasive  in  attempting 
to  use  new  data  to  support  decision. 

Benchmark  four 


Showed  considerable  skill  in  persuading  others  -  both  in  getting 
ideas  accepted  and  changing  their  positions.  Convinced  the  group  to 
accept  a  set  of  criteria  which  was  then  used  to  support  own  department 
in  City  Council  Exercise.  Used  minimum  data  supported  by  logical 
arguments  to  sell  decision  on  Research  Budget.  Very  persuasive  in 
countering  assessor's  arguments  relatively  quickly. 


Figure  6-5 

DEFINITION  AND  BENCHMARKS :  PERSUASIVENESS 
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Flexibility:  To  what  extent  can  this  individual  adjust  to  new  situations 

easily? 

Additional  points  to  consider: 

Reaction  to  new  information 

Variation  in  behavior  and  style  with  different  people 
Willingness  to  compromise 
Variation  in  tactics 
Acceptance  of  other's  ideas 

Ability  to  change  roles  (e.g.,  from  leader  to  follower) 

Benchmark  Two 


Rarely  altered  method  or  style  in  spite  of  often  being  alone 
or  in  the  minority  on  issues.  Insisted  on  gettin  entire  grant  for 
own  department  in  City  Council  even  though  others  presented  good 
arguments.  In  Research  Budget,  stuck  with  pre-planned  questions, 
even  with  new  facts. 


Benchmark  Three 


Somewhat  receptive  to  new  ideas.  Style  is  relaxed  and  easy¬ 
going  and  changes  very  little.  In  Research  Budget,  strayed  very  little 
from  pre-planned  questions,  but  shifted  position  based  on  later  argu¬ 
ments  from  assessor.  As  a  City  Council  member  was  willing  to  listen 
to  others'  proposals,  and  was  willing  to  compromise  with  only  slight 
reluctance.  In  discussion  the  management  Problems,  went  along  with  the 
majority  most  of  the  times 


Benchmark  Four 


Shows  a  lot  of  ability  to  adapt  to  new  situations  and  new  infor¬ 
mation.  In  the  Research  Budget,  was  quick  to  pick  up  leads  and  follow 
them  through  -  even  though  this  varied  from  planned  approach.  Demon¬ 
strated  flexibility  in  examining  others'  City  Council  proposals  and 
in  proposing  alternative  solutions  and  projects  -  seemed  to  always 
have  a  back-up  plan.  Came  on  very  strong  at  first  on  Supervisory 
Task  Force.  When  rejected  by  the  group,  toned  down  approach  and 
made  several  inputs  which  were  accepted  by  the  group. 


Figure  6-6 

DEFINITION  AND  BENCHMARKS :  FLEXIBILITY 
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Empathy:  To  what  extent  does  this  individual  show  concern  for  others? 

Additional  points  to  consider: 

Sensitivity  or  awareness  of  others'  feelings 
Respect  while  others  are  speaking 

Oo  others  feel  comfortable  around  this  individual? 


Benchmark  Two 

Though  not  abrasive,  displays  very  little  tolerance  for  those 
who  were  poor  performers.  Extends  normal  courtesy  to  others.  Very 
little  concern  for  others  surfaced  in  any  of  the  exercises. 


Benchmark  Three 


Attentive  to  others  remarks,  feels  job  can  get  accomplished 
without  being  unduly  harsh  on  anyone.  In  Management  Problem  preferred 
to  compromise  rather  than  alienate  anyone.  Displays  an  awareness  of  the 
importance  of  people  issues.  During  Interview  expressed  an  interest  in 
working  with  people  and  a  concern  for  both  family  and  friends  as  well  as 
the  job.  In  the  Supervisory  Task  force  was  not  completely  diplomatic. 

At  times  exhibited  some  impatience  with  others. 


Benchmark  f our 


Exhibits  a  genuine  concern  for  others.  Listens  attentively  to 
others  without  interrupting.  Displays  above  average  sensitivity.  Likes 
working  with  and  assisting  others.  Ouring  management  Problems  expressed 
concern  for  employee  morale.  In  the  Supervisory  Task  Force  introduced 
the  idea  of  human  relations,  was  willing  to  change  to  accomodate  feelings 
of  others.  Solicited  comments  from  all  group  members. 


f igure  6-7 


DEFINITION!  AND  BENCHMARKS :  EMPATHY 


Stress  Tolerance:  To  what  extent  does  this  individual  exhibit 
stability  of  performance  under  pressure,  opposition,  or  other 
difficult  conditions. 

Additional  points  to  consider: 

Poise 

Display  of  nervousness  or  irritation 
Change  in  speech  or  behavior 

Note:  Stress  is  inherent  in  an  assessment  center  environment.  "The 
stress  in  an  assessment  center  comes  from  the  other  candidates  and 
from  the  quantity  and  nature  of  the  material"  (Norton,  and  Edinger, 
1978:13).  If  an  individual  "can  survive  the  exercise  as  presented 
without  indicating  stress,  he  should  ge  a  good  rating  cn  stress 
tolerance"  (Moses,  1974). 


Benchmark  Two 


Performance  is  degraded  under  stress  or  conflict.  Cutward 
display  of  nervousness  -  tapping  pencil,  biting  fingernails,  strained 
voice,,  etc.  Especially  nervous  in  group  CAercises.  Stress  was 
apparent  in  speech  during  City  Council  formal  presentation.  Always 
looked  dawn,  never  at  group  during  discussion  cn  Management  Problem, 
became  t'rustraten  and  face  was  flushed  when  new  data  was  introduced 
in  Research  Budget 


Benchmark  Three 


Outward  appearance  of  calmness.  Performance  is  not  degraded 
seriously  although  stress  is  obviously  felt.  In  City  Council,  remained 
calm  even  though  own  proposal  was  not  receiving  much  attention  - 
slight  tendency  to  withdraw.  Relaxed  and  active  participant  in  solving 
Management  Problems  until  time  pressure  was  felt.  Expressed  frustration 
by  making  negative  comments  to  others.  Did  not  appear  to  be  bothered 
by  stress  during  Research  EudQet,  even  when  challenged  by  the  assessor. 
At  times,  voice  shows  a  little  strain,  but  not  too  apparent. 

Benchmark  Four 


Went  into  all  exercises  with  relaxed  attitude  and  maintained 
composure  throughout.  No  outward  signs  of  stress.  In  Research  Budget 
was  completely  poised  in  both  the  questioning  and  presentation  period, 
reacting  quickly  and  calmly  to  questions  and  comments.  The  only  stress 
shown  was  when  told  of  missing  a  piece  of  data  -  showed  some  irritation 
but  recovered  quickly.  Not  disturbed  by  conflict  during  City  Council. 
Instead,  worked  hard  in  seeking  compromisies •  No  signs  of  stress  in 
discussion  of  Management  Problems  -  always  spoke  in  calm,  clear  voice. 


Figure  6-0 

DEFINITION  AND  BENCHMARKS :  STRESS  TOLERANCE 
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Risk  Taking;  To  what  extent  is  this  individual  willing  to  risk 
negative  consequences  in  order  to  achieve  objectives? 

Additional  Points  to  consider: 

Willingness  to  state  a  position  or  take  a  stand 
Willingness  to  make  a  decision  based  on  limited  information 
Willingness  to  take  an  unpopular  position 
Willingness  to  assume  responsibility 

B enchmark  Two 


i ery  cautious  about  expressing  an  opinion.  Usually  does  not 
take  a  very  firm  stand.  Coes  not  get  deeply  involved.  Is  not  willing 
to  take  a  risk  to  achieve  objectives.  During  interview  indicated 
dislike  tor  ambiguous  work  situations.  During  Management  Problems 
lack  of  participation  indicated  reluctance  to  risk  criticism  of  others. 

Benchmark  Three 


■.'lilling  to  disagree  with  others  and  to  take  an  unpopular  position 
„.oup  exercises.  Did  not  indicate  desire  to  take  on  added  respon- 
sj.oiii'.ies.  in  Management  Problems  often  made  comments  and  disagreed 
with  others.  Ouring  City  Council  made  a  fairly  safe  decision  to  com¬ 
promise. 


Benchmark  Four 


In  all  exercises  was  willing  to  take  either  the  risk  or  the 
responsioility  to  get  the  job  done.  Quick  to  take  action,  wants  to 
cet  things  done,  not  reluctant  to  make  decisions  and  justify  it. 
During  the  Interview  indicated  desire  to  do  innovative  things.  In 
Research  Budget  was  willing  to  make  decision  with,  minimum  amount  of 
data. 


Figure  6-9 

DEFINITION  ANO  BENCHMARKS:  RISK  TAKING 
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Acquiring  Information;  To  what  extent  is  this  inbi\ 
obtain  information? 


j!  able  to 


Additional  Points  to  consider: 

Number  of  questions  asked. 

How  aggressive  is  questioning? 

Is  questioning  adjusted  to  data  received  in  answers 
to  previous  questions? 

How  much  information  is  obtained? 

Follow-through  on  questions. 

Ability  to  determine  what  information  is  needed. 

Note:  See  Figure  6-16  for  relationship  between  Decision  Waking,  Problem 
Solving,  and  Acquiring  Information. 

Benchmark  Two 

Information  gathering  is  shallow  in  depth  and  general  in  nature. 
Oniy  responded  to  questions  asked  by  assessor  in  Tnterview.  Although 
given  the  opportunity,  sought  no  information  from  the  assessor.  Curing 
Research  Budget  stuck  strictly  to  original  set  of  questions  and  obtained 
only  minimal  data.  In  City  Council  Exercise,  asked  very  few  questions 
of  others. 

Benchmark  Three 

Fairly  good  at  ootaining  information,  however,  does  nr'  ?np«Mr 
to  recognize  its  significance  for  follow-up  questions.  Used  planm 
period  in  Researcn  Budget  to  organize  a  logical  set  of  questions. 

Revised  leads  uncovered  during  questioning  but  frequently  got  =  .oe- 
tracked.  Asked  pertinent  questions  of  other  City  Council  members  to 
obtain  information  used  to  support  own  proposal.  When  a  weak  point 
was  found,  #3  followed  it  up.  During  the  Interview  asked  very  few 
questions,  but  wanted  to  review  the  background  questionnaire  to  make 
sure  nothing  was  missed. 

Benchmark  Four 

Well-planned  approach  to  obtaining  a  large  amount  of  information 
in  a  limited  amount  of  time.  In  the  Research  Budget  has  a  rapid-fire 
questioning  technique  and  pursued  all  leads  and  clues  to  a  conclusion. 

In  the  city  council  discu:  ion,  asked  many  questions  and  refused  to 
accept  half  answers  but  aggressively  sought  pertinent  facts.  Also 
picked  out  pertinent  facts  in  problem  statement.  Asked  several  questions 
about  the  assessment  center  process  and  use  of  results  during  the  inter¬ 
view. 

Figure  6-10 

DEFINITION  AND  BENCHMARKS:  ACQUIRING  INFORMATION 
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Organizing  Skills;  To  what  extent  does  this  individual  plan  and  organize 
his/her  own  activities  and  those  of  others? 

Additional  points  to  consider: 

Are  oral  and  written  reports  logical  and  understandable? 

Efficient  use  of  time? 

Does  the  individual  set  goals  and  work  toward  them? 

Are  activities  structured? 

Are  priorities  assigned  to  different  activities? 

Benchmark  Two 


demonstrated  an  average  ability  to  organize  own  work,  but 
ineffective  in  organizing  group  activities.  Oral  presentation 
during  City  Council  reflected  no  strategy.  In  Research  Budget, 
available  facts  were  not  used  to  support  arguments.  Organizational 
Problem  input  was  not  well  organized,  consisting  mainly  of  random 
thoughts. 


Benchmark  Three 


Moderately  well  organized  in  group  exercises.  In  City  Council 
made  several  attempts  at  organizing  the  group.  Suggested  spokesman 
be  selected  and  criteria  for  judging  proposals  be  established.  In 
management  Problems  suggested  time  limit  be  set  for  each  problem  and 
priorities  be  established.  Research  8udget  report  was  organized 
into  sections. 


Benchmark  four 


Well  organized  in  both  individual  and  group  exercises.  Showed 
ability  to  organize  thoughts  and  formulate  ideas  throughout  discussion 
in  examining  and  questioning  others  proposals  in  City  Council.  Oral 
presentation  was  logical,  pertinent  and  organized  to  achieve  objectives. 
In  the  Research  Budget,  questions  were  logical  and  comprehensive.  List 
of  answers  were  complete  and  reports  well  organized  to  present  findings. 
Used  a  good  outline  in  the  Organizational  Problem  to  organize  approach. 


figure  6-11 
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Problem  Solving:  To  what  extent  can  this  individual  identify  problems 
and  find  solutions? 

Additional  points  to  consider: 

Ability  to  sift  out  significant  information  or  important 
facets  of  a  problem  and  disregard  non-essentials. 

Ability  to  size  up  a  situation  quickly,  including  the 
people  involved. 

Are  alternative  solutions  identified  and  examined? 

Amount  of  participation  in  group  problem  solving  situations. 

Note:  See  Figure  6-1C  for  relationship  between  Decision  Waking, 
Problem  Solving  and  Acquiring  Information. 

Benchmark  Two 


Mas  difficulty  in  grasping  problems  and  understanding  significance 
of  data.  Tends  to  support  other  approaches  rather  than  seek  a  new  sol¬ 
ution.  Snows  little  participation  and  little  effect  on  solving  a  group 
problem.  In  tne  Management  Problems  displayed  some  understanding  of 
problem  but  was  never  able  to  arrive  at  solution.  During  Research 
Budget  did  not  grasp  problem  or  understand  significance  of  data.  In 
Supervisory  Tack  force  was  more  concerned  with  resolving  minor  disputes 
than  with  the  group  problem. 


Benchmark  Three 

Considers  pertinent  inputs  to  a  problem  and  arrives  at  a  solution. 
Plans  material  and  assesses  facts  and  data  fairly  well.  During  Manage¬ 
ment  Problems,  developed  snl'.tions  to  problems  and  brought  them  out 
after  considerable  thought  .o.d  listening  to  others.  In  Research  Budget 
could  not  completely  solve  problem  because  of  insufficient  data.  In 
Supervisory  Task  Force,  generally  took  ideas  of  others  and  refined 
them  toward  a  group  solution. 


Benchmark  Four 


Uses  rational  approaches  based  on  facts,  considers  options,  and 
easily  arrives  at  good  solutions  to  the  problems  posed.  Combines  others 
suggestions  as  necessary  to  arrive  at  solutions.  Readily  understood 
problems  in  Management  Problems  and  presented  accepted  solutions  to 
two.  Recommend  logical  first  step  to  the  Research  Budget  problems.  In 
Supervisory  Task  Force,  identified  barriers  to  agreement  and  proposed 
alternative  methods  to  resolve. 


Figure  6-12 

DEFINITION  AND  BENCHMARKS:  PROBLEM  SOLVING 
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Decision  makings  To  what  extent  can  this  individual  make  decisions  based 
on  sound  rationale? 

Additional  points  to  consider: 

Ability  to  determine  key  issues 

Is  decision  making  process  logical  and  methodical? 

Are  all  inputs  considered? 

Note:  See  Figure  6-16  for  relationship  between  Decision  Waking, 

Problem  Solving,  and  Acquiring  Information 

Benchmark  Tuio 

Decision  were  tentative.  Rarely  makes  a  firm  statement.  When 
a  decision  is  made,  the  quality  is  questioned  since  it  is  readily 
changed  to  accomodate  challenges  and/or  additional  information.  In 
Research  Budget,  didn't  feel  adequate  information  was  available  until 
the  very  end  of  the  exercise.  Reached  no  decision  in  the  Organizational 
Problem.  Based  any  decisions  on  assumptions  and  feelings  rather  than 
factual  information. 


Benchmark  Three 


Willing  to  make  decisions.  In  group  exercises,  decisions  were 
fairly  logical  and  convincing.  Also  decisive  in  one-on-one  situation, 
but  decisions  not  always  based  on  sound  logic.  In  Management  Problems, 
positively  stated  decision  regarding  one  of  the  four  problems.  Appears 
able  to  make  a  decision,  but  sometimes  reluctant  to  put  it  before  a 
group.  Will  usually  support  decisions  in  face  of  challenges.  In  City 
Council  decided  to  withdraw  department. 

Benchmark  Four 


Makes  decisions  quickly  which  are  reasonable  and  supportable. 
Does  not  hesitate  to  change  position  if  additional  information  is  ob¬ 
tained  and  justifies  a  reversal.  Decisions  based  on  good  insight  and 
facts.  In  City  Council  was  willing  to  make  a  decision  with  available 
facts  well  before  time  expired.  In  the  Research  Budget  made  good 
decision  supported  with  solid  rationale.  Used  positive  approach, 
ignoring  irrelevant  data. 

Figure  6-13 

DEFINITION  AND  BENCHMARKS:  DECISION  MAKING 
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Written  Communication  5killsi  To  what  extent  can  this  individual 
effectively  express  ideas  in  a  well-organized,  grammatically  correct 
written  form? 

Additional  po.  .ts  to  consider: 

Are  sentences  and  paragraphs  well-structured? 

Is  the  report  easy  to  read  and  understand? 

Is  the  report  clear  and  concise,  or  are  excess  words  used? 

Handwriting  should  not  be  a  factor  unless  it  distracts  from 
the  message 

Does  the  report  have  an  introduction,  main  points,  and  conclusion? 

Benchmark  Two 

Writing  lacks  styie  and  organization.  Written  mostly  in  long 
rambling  sentences  with  little  substance.  Transitions  between  sentences 
and  paragraphs  poor.  Same  incomplete  sentences.  What  was  written  in 
the  Research  Budget  was  merely  a  restatement  of  the  original  problem 
and  the  information  collected. 


Benchmark  Three 


Written  exercises  were  fairly  clear  and  logical.  Usually 
easy  to  read  and  coherent.  Had  no  conclusions  in  Organizational 
Problem  report.  Narrative  report  in  Research  Budget  pulled 
together  the  dat3  obtained.  Good  grammar,  only  a  few  instances  of 
improper  sentence  structure  or  misspelling.  Satisfactory  organ¬ 
ization,  but  did  not  include  all  available  facts  in  written  report 
for  Research  Budget. 


Benchmark  Four 


Uses  logical  organization  and  better  than  average  sentence 
structure,  material  is  well  though  out  and  easily  understood.  Used 
facts  effectively  in  Research  Budget  to  bring  point  across.  Contains 
very  little  superfluous  material.  Writing  is  complete,  factual, 
displays  good  vocabulary  with  very  few  errors  in  spelling 


F igure  6-14 

DEFINITION  AND  BENCHMARKS :  WRITTEN  COMMUNICATION  SKILLS 
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Dimension 
Leadership 
Persuasiveness 
Force  fulness 
Energy 

RELATIONSHI 


Key  Point 

Gets  others  to  accomplish  task 
Gets  others  to  accept  ideas 
Gets  others  to  pay  attention 
Level  of  activity 

Figure  6-15 

P  BETWEEN  LEADERSHIP,  PERSUASIVENESS, 
FORCEFULNESS,  AND  ENERGY 
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Dimension 


Key  Point 


Decision  making 


Selecting  alternatives  best  supported 
by  the  information  available 


Problem  Salving 


Evaluating  and  interpreting 
information 


Acguiring  Information 


Determining  information  needs  and 
identifying  sources 


Figure  6-16 

RELATIONSHIP  BETWEEN  OECISION  MAKING,  PROBLEm  50LVING 
AND  ACQUIRING  INFORMATION 
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(Dost  assessment  centers  achieve  these  results  by  means  of  the 
feedback  interview  for  which  the  assessors  receive  special  training.  The 
content  of  the  interview  is  based  on  a  report  written  by  the  assessor 
giving  the  interview.  The  report  typically  provides  a  rating  for  each 
of  the  dimensions  evaluated,  an  overall  rating  of  the  candidates  poten¬ 
tial  for  success,  and  a  narrative  description  of  the  candidates  strengths 
and  weaknesses.  A  center  used  for  management  development  would  emphasize 
future  needs  and  would  provide  a  plan  for  correcting  deficiencies. 

Appendix  £  provides  a  suggested  outline  for  the  assessor  feedback  inter¬ 
view.  It  is  cased  on  two  different  outlines  used  with  the  ASO  Center 
and  one  outline  developed  by  Salter  and  Qlmstead  (19?4:2)  shown  in 
figure  4-1. 

Even  before  the  feedback  interview,  assessees  usually  complete 
the  assessment  center  with  a  good  Idea  of  how  they  have  done.  Just 
the  experience  of  going  through  the  assessment  center  should  lead  to 
increased  self-insight  into  their  own  strengths  and  weaknesses. 

According  to  Byham  (1971:12)  such  self-insight  is  usually  fairly  accurate. 


Data  Catherine 


The  updated  ASO  Center  can  be  used  to  further  investigate  the 


usefulness  of  the  assessment  center  for  management  development.  Any 
further  investigation  should  be  directed  toward  determining  (1)  the 
validity  of  the  ASO  Center  using  Air  Force  officers  as  assessors  and 
assessees,  and  (2)  the  usefulness  to  individual  Air  Force  officers  in 


terms  of  providing  feedback  for  management  development. 


At  the  beginning  of  this  research  effort,  it  was  envisioned  that 


a  pilot  center  evaluating  24  or  more  A5D  Captains  be  run  to  gather  the 
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information  needed  to  answer  those  questions.  Majors  and  Lieutenant 
Colonels  would  be  trained  to  serve  as  the  assessors. 

Empirical  validation  similar  to  that  presented  in  Chapter  V 
should  be  accomplished  and  compared  with  the  present  results  to  detect 
any  differences  or  problems  in  the  assessment  center  operation.  In 
addition,  feedoack  should  be  solicited  from  the  assessees  in  order  to 
obtain  their  reactions  to  the  assessment  center.  A  questionnaire 
developed  to  assist  in  this  process  is  shown  in  Figure  6-17.  Appendix 
0,  referenced  previously,  served  as  the  basis  for  this  questionnaire. 
Finally,  the  assessors  themselves  should  be  questioned  to  determine 
their  reactions  to  the  assessment  center.  The  questionnaire  shown  in 
Figure  6-10  was  developed  for  this  purpose. 

Summary 

This  chapter  has  presented  the  results  of  efforts  to  modify 
the  1974-75  A5D  Assessment  Center  design  tor  possible  use  as  a  pilot 
assessment  canter  for  management  development  of  Air  Force  officers. 

The  modification  included  expanded  definitions,  specially  developed 
benchmarks,  an  outline  for  providing  assessor  feedback,  and  outlines 
for  obtaining  reactions  from  both  assessees  and  assessors. 
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1.  Did  you  have  any  prior  knowledge  concerning  the  assessment  center 
method? 

2.  How  much  previous  counseling  have  you  had  concerning  your  develop¬ 
mental  needs  in  the  area  of  management  skills? 

3.  Were  the  exercises  realistic  compared  to  your  view  of  an  Air  Force 
manager's  job? 

4.  Do  you  feel  that  the  dimensions  used  in  this  assessment  center 
accurately  reflect  the  management  skills  needed  by  Air  Force 
managers?  What  would  you  delete?  Ulhat  would  you  add? 

5.  Would  you  consider  the  assessment  center  worthwhile  without 
the  final  feedback  interview? 

6.  In  your  opinion,  were  the  assessors  well-qualified? 

7.  Do  you  agree  with  the  feedback  you  received  in  the  finai  interview? 
If  not,  please  explain. 

8.  Oid  you  recognize  your  development  needs  prior  to  the  assessment 
center? 

9.  Oid  you  recognize  your  strong  points  prior  to  the  assessment  center’’ 

10.  Do  you  believe  the  assessment  center  method  is  of  value  as  a 
management  development  tool  for  Air  Force  officers? 

11.  Would  you  recommend  the  assessment  center  to  others  as  a  develop¬ 
mental  tool? 

12.  Oo  you  think  the  assessment  center  should  be  used  throughout  the 
Air  Force  For  officer  develops, ent? 


Figure  6-17.  Assessee  Reaction  Questionnaire 
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1.  In  your  opinion,  is  the  assessment  center  an  effective  counseling 
technique? 

2.  Oo  you  find  it  easier  to  give  more  objective  feedback  in  the 
assessment  center  environment? 

3.  Do  you  think  all  supervisors  should  be  required  to  serve  as 
assessors? 

A.  Oo  you  think  all  supervisors  should  receive  assessor  training, 
even  if  they  do  not  ever  serve  as  assessors? 

5.  Were  the  exercises  realistic  compared  to  your  view  o'  an  Air 
Force  manager's  job? 

6.  Oo  you  feel  that  the  dimensions  used  in  this  assessment  center 
accurately  reflect  the  management  skills  needed  by  Air  Force 
managers?  What  would  you  delete?  UJhat  would  you  add? 

7.  Co  you  feel  that  your  participation  as  an  assessor  will  make  you 
a  better  supervisor?  If  so,  in  hat  ways? 

S.  Was  the  training  you  received  adequate?  If  not,  please  explain. 

9.  Oo  you  think  the  assessment  center  should  be  used  throughout 
the  Air  Force  for  officer  development? 


Figure  6-18.  Assessor  Reaction  Questionnaire 
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ChAFTER  VII 


SUMIYIAR Y  AND  CONCLUSIONS 

Summary 

"Officer  Development"  is  a  broad  term  encompassing  both  tech¬ 
nical  and  management  training  throughout  an  officer’s  career.  The 
Air  force  has  adopted  both  formal  and  informal  methods  of  development. 
The  focus  of  this  thesis  has  been  on  that  portion  of  development  con¬ 
cerned  with  management  skills.  Weaknessess  have  been  identified  in 
the  informal  methods  of  officer  management  development,  particularly 
regarding  feedback  to  officers  concerning  their  management  skills. 

The  assessment  center  was  investigated  as  a  method  of  identifying 
strengths  and  weaknesses  in  an  officer's  management  skills. 

Assessment  centers  have  been  used  previously  by  the  military. 
They  were  used  by  the  Germans  in  Ulorld  Uiar  I  and  the  British  in  World 
War  II  for  officer  selection,  and  by  the  United  States  in  World  War  II 
for  the  selection  of  intelligence  agents.  Since  World  War  II,  the 
assessment  center  has  received  limited  use  by  the  military  service; 
however,  both  the  Army  and  the  Air  Force  have  conducted  trial  centers 
and  performed  research  as  well.  In  American  industry  tney  have  been 
used  for  a  number  of  different  purposes  since  1956.  These  uses  include 
employment,  early  identification,  placement,  advancement,  development, 
affirmative  action,  and  training.  The  principal  use  has  been  to  aid 


in  selection  of  personnel  for  advancement  or  promotion;  however,  there 
is  a  growing  trend  toward  combining  assessment  and  development. 

The  ASO  Management  Assessment  Center,  operated  at  Ulright- 
Patterson  AF3  in  197A-T975,  was  examined  for  application  to  management 
development  of  Air  Force  officers.  Its  dimensions  included  most  of  the 
criteria  identified  as  important  to  officer  development.  A  study  of 
this  center  revealed  that  it  appears  to  be  valid  with  regard  to  content, 
predictive  ability  and  internal  design.  The  only  problem  appeared  tc 
stem  from  inadequate  training  of  assessors  in  observing  and  recording 
dimensions.  Other  conclusions  concerning  the  validity  study  are  dis¬ 
cussed  later  in  this  chapter. 

After  conducting  the  validity  study  and  Determining  that  the 
ASO  Management  Assessment  Center  could  be  used  for  officer  development, 
the  ASD  model  was  updated  for  future  application.  This  update  involves 
three  areas.  First,  a  program  for  training  assessors  was  described. 

To  aid  in  training,  expanded  dimension  definitions  .  nq  benchmarks 
based  on  assessor  reports  were  prepared  to  aid  in  s tandsrdization  of 
ratings.  Second,  the  feedback  of  assessment  results  to  assessees  was 
discussed  cine*,  this  is  perhaps  the  most  crucial  part  of  a  development 
center.  Ar  cjtline  was  suggested  to  guide  the  feedback  interview, 
fina.'.l/,  l ne  approach  to  gathering  additional  data  from  assessees  and 
assessors  abcuv  the  utility  of  this  method  for  management  development 

WPS  OiSCUSSP^. 


Conclusions 


1.  There  is  a  need  for  improved  officer  management  development 


programs  particularly  in  the  area  at  feedback  on  strengths  and 
weaknesses  in  management  skills. 

2.  The  A5D  Management  Center  is  a  valid  model  for  assessing 

the  strengths  and  weaknesses  of  Air  force  officers  assigned  to  scientific 
and  engineering  positions  at  ASG,  although  some  problems  were  detected 
in  the  measurement  of  several  dimensions.  Detailed  conclusions  are 
listed  below: 

a.  Acquiring  information  should  be  excluded  from  the 

Interview. 

b.  The  organizational  problem  exeroise  needs  to  be  examined 
for  possible  problems  in  the  measurement  of  dimensions  within  that 
exercise ■ 

c.  Measurement  problems  exist  for  the  stress  tolerance 
dimension.  This  was  indicated  first  by  the  low  inter-r3ter  reliability 
and  second  by  reviewing  the  assessor  worksheets. 

d.  Evidence  indicates  that  empathy  js  a  dimension  independent 
of  all  others  and  has  little  bearing  on  performance  within  the  assessment 
center „  It  is  felt  that  empathy  can  be  eliminated  from  the  dimensions 
evaluated. 

3.  Training  of  assessors  in  the  accurate  evaluation  of  indi¬ 
viduals  in  the  different  dimensions  is  the  key  element  in  the  effective¬ 
ness  or  the  assessment  center.  Precise  and  differentiable  definitions 
and  benchmarks  for  calibrating  assesses  behavior  in  a  specific  dimension 
are  essential. 

A.  Although  a  vast  amount  of  knowledge  is  available  on  assess¬ 
ment  centers  and  even  though  the  military  has  conducted  considerable 
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research,  more  data  is  needed  in  order  to  Fully  evaluate  the  effective¬ 
ness  of  the  assessment  center  in  the  Air  force  for  development  and 
other  purposes. 


Recommendations 

1.  Conduct  a  pilot  assessment  center  as  described  in  Chapter  VI. 
The  primary  purpose  of  this  effort  should  be  to  assess  the  reaction  of 
Air  Force  officers  to  the  use  of  the  assessment  center  as  a  management 
development  method. 

2.  Investigate  other  criterion  variables,  such  as  the  results 
of  a  personal  interview  on  job  performance  with  the  assessee's 
immedia.e  supervisors,  for  determining  the  predictive  validity  of 
Ootn  tne  ASO  Center  and  the  pilot  assessment  center. 

3.  Accomplish  additional  research  to  determine  what  differences 
if  any,  exist  in  dimensions  critical  to  success  for  civilian  managers 
versus  Air  force  managers.  This  should  include  an  examination  of 
whether  or  not  an  assessment  center  should  be  designed  for  general 
management  development  of  all  Air  Force  officers  or  if  it  needs  to  be 
specifioally  designed  for  a  target  job. 

4.  Investigate  the  possible  implementations  of  an  Air  Force 
wide  assessment  center  program  for  management  development.  This  should 
include  an  examination  of  such  areas  as:  frequency,  cost,  location, 
manning,  and  potential  candidates. 
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1.  Rationale  for  Assessment  Center  Standards 


The  rapid  growth  in  the  use  of  the  Assessment  Center  Method  in 
recent  years  has  resulted  in  a  proliferation  of  applications  in 
a  variety  of  organizational,  educational  and  governmental  settings. 
Serious  concerns  have  been  raised  by  many  interested  parties  which 
reflect  a  need  for  a  set  of  minimal  professional  standards  for 
users  of  this  technique.  These  standards  should: 

-  define  what  is  meant  by  an  assessment  center 

-  describe  minimal  acceptable  practices  concerning: 

—  organizational  support  for  assessment  operations 

—  assessor  tr  ling 

--  informed  consent  on  the  part  of  participants 

—  use  of  assessment  center  data 

—  validation  issues 

These  standards  are  not  design  to  prescribe  specific  practices. 
Neither  do  these  standards  if  /  way  endorse  a  specific  assessment 
center  format  or  specific  assessment  techniques.  Rather,  we  have 
attempted  to  provide  general  principles  which  can  be  adapted  to 
meet  existing  and  future  applications.  The  reader  should  keep  in 
mind  the  spirit  by  which  these  standards  were  written:  as  an  aid  to 
the  assessment  center  user,  rather  than  as  a  set  of  restrictive 
prohibitions. 


2.  Assessment  Center  Oe fined 


To  be  considered  as  an  assessment  center,  the  following  minimal 
requirements  must  be  met: 

1.  Multiple  assessment  techniques  must  be  used.  At  least 
ens  of  these  techniques  must  be  a  simulation. 

A  simulation  is  an  exercise  or  technique 
designed  to  elicit  behaviors  related  to 
dimensions  of  performance  on  the  job  by 
requiring  the  participant  to  respond  be- 
haviorally  to  situational  stimuli.  The 
stimuli  present  in  a  simulation  parallel 
or  resemble  stimuli  in  the  work  situation. 
Examples  of  simulations  include  group 
exercises,  in-basket  exercises  and  tact 
finding  exercises. 
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2.  Multiple  assessors  must  be  used.  These  assessors  must 
receive  training  prior  to  participating  in  a  center. 

3.  Judgements  resulting  in  an  outcome  (i.e.,  recommendation 
for  promotion,  specific  training  or  development)  must  be 
based  on  pooling  information  from  assessors  and  techniques. 

4.  An  overall  evaluation  of  behavior  must  be  made  by  the 
assessors  at  a  separate  time  from  observation  of  behavior. 

5.  Simulation  exercises  are  used.  These  exercises  are 
developed  to  tap  a  variety  of  predetermined  behaviors 
and  have  been  pretested  prior  to  use  to  insure  that  the 
techniques  provide  reliable,  objective  and  relevant 
behavioral  information  for  the  organization  in  question. 

6.  The  dimensions,  attributes,  characteristics  or  qualities 
evaluated  by  the  assessment  center  are  determined  by  an 
analysis  of  relevant  job  behaviors. 

7.  The  techniques  used  in  the  assessment  center  are  designed 
to  provide  information  which  is  used  in  evaluating  the 
dimensions,  attributes  or  qualities  previously  determined. 

In  summary,  an  assessment  center  consists  of  a  standardized  evaluation 
of  behavior  based  on  multiple  inputs.  Multiple  trained  observers 
and  techniques  are  used.  Judgements  about  behavior  are  maae,  in 
part,  from  specially  developed  assessment  simulations. 

These  judgements  are  pooled  by  the  assessors  at  an  evaluation 
meeting  during  which  all  relevant  assessment  data  are  reported 
and  discussed,  and  the  assessors  agree  on  the  evaluation  of  the 
dimensions  and  any  overall  evaluation  that  is  made. 

The  following  kinds  of  activities  do  not  constitute  an  assessment 
center! 

1.  Panel  interviews  or  a  series  of  sequential  interviews 
as  the  sole  technique. 

2.  Reliance  on  a  specific  technique  (regardless  of  whether 
a  simulation  or  not)  as  the  sole  basis  for  evaluation. 

3.  Using  only  a  test  battery  composed  of  a  numoer  of  pencil 
and  paper  measures,  regardless  of  whether  the  judgements 
are  made  by  a  statistical  or  judgmental  pooling  of  scores. 

4.  Single  assessor  assessment  (often  referred  to  as  individual 
assessment)  -  measurement  by  one  individual  using  a  variety 
of  techniques  such  as  pencil  and  paper  tests,  interviews, 
personality  measures  or  simulations. 


I 
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5. 


The  use  of  several  simulations  with  more  than  one  assessor 
where  there  is  no  pooling  of  data,  i.e.,  each  assessor 
prepares  a  report  on  performance  in  an  exercise,  and  the 
individual  reports  (unintegrated)  are  used  as  the  final 
product  of  the  center. 

6.  A  physical  location  labeled  as  an  "assessment  center" 
which  does  not  conform  to  the  requirements  noted  above. 

3.  Organizational  Support  for  Assessment  Center  Operations 

The  assessment  center  should  be  administered  in  a  professional 
manner  with  concern  for  the  treatment  of  individuals,  accuracy 
of  results  and  overall  quality  of  the  operation.  Assessment 
centers  should  be  incorporated  as  part  of  a  total  system  rather 
than  as  a  process  that  operates  in  a  vacuum.  Considerable  care 
and  planning  should  precede  the  introduction  of  an  assessment 
center.  Policy  statements  concerning  assessment  operations 
should  be  formally  developed  and  agreed  upon  by  the  organization. 
Minimal  considerations  in  developing  this  policy  should  include: 

1.  The  population  to  be  assessed. 

2.  The  purpose  of  assessment. 

3.  The  kinds  of  people  who  will  serve  as  assessors. 

4.  The  type  of  training  they  receive  and  who  is  to 
provide  it. 

5.  The  responsibility  for  administration  of  the  center. 

6.  Specific  restrictions  concerning  who  is  to  see  the 
assessment  data,  and  how  they  are  tD  be  used. 

7.  Procedures  for  collection  of  data  for  research  and 
program  evaluation. 

8.  feedback  procedure  to  participants  and  management. 

9.  Expected  "life"  of  assessment  center  data  -  i.e.,  the 
length  of  time  assessment  center  data  will  be  kept 

in  the  files  and  used  for  decision  making  purposes. 

10.  The  professional  qualifications  (including  relevant 
training)  of  the  individual  or  individuals  initially 
responsible  for  developing  the  center. 
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Assessors  should  receive  sufficient  training  to  enable  them  to 
evaluate  intelligently  the  behaviors  measured  in  the  center. 
"Sufficient  training"  will  vary  from  organization  to  organization 
and  is  a  function  of  many  factors  including: 

-  The  length  of  time  an  individual  serves  as  an  assessor. 

-  The  frequency  of  individual  participation  as  an  assessor. 

-  The  amount  of  time  devoted  to  assessor  training. 

-  The  qualification  and  expertise  uf  the  assessment  center 
trainer. 

-  The  assessment  experience  of  other  members  of  the 
assessment  staff. 

-  The  use  of  professionals  (i.e.,  licensed  or  certified 
psychologists)  as  assessors. 

The  above  list  is  illustrative  of  the  many  issues  related  to 
assessor  training.  There  is  more  variability  in  this  area  than 
in  any  other  section  of  the  stanoards. 

While  we  do  not  wish  to  establish  minimal  standards  concerning 
the  number  of  hours  of  assessor  training  needed,  it  is  difficult 
to  imagine  assessors  functioning  effectively  with  only  a  one  or 
two  hour  orientation  prior  to  serving  as  an  assessor.  However, 
whatever  the  approach  to  assessor  training,  the  essential  goal  is 
attaining  accurate  assessor  judgments.  A  variety  of  training 
approaches  can  be  used,  as  long  as  it  can  be  demonstrated  that 
accurate  assessor  judgments  are  obtained.  The  following  minimum 
training  is  required* 

1.  Knowledge  of  the  assessment  techniques  used.  This  could 
include,  for  example,  the  kinds  of  behaviors  elicited  by 
each  technique,  relevant  dimensions  to  be  observed, 
expected  or  typical  behaviors,  examples  or  samples  of 
actual  behaviors,  etc. 

2.  Knowledge  of  the  assessment  dimensions.  This  could 
include,  for  example,  definitions  of  dimensions,  rela¬ 
tionship  to  other  dimensions,  relationship  to  job 
performance,  examples  of  effective  and  ineffective 
performance,  etc. 

3.  Knowledge  of  behavior  observation  and  recording  including 
the  forms  used  by  the  center 


4.  Knowledge  of  evaluation  and  rating  procedures  including 
how  data  are  integrated  by  the  assessment  center  staff. 

5.  Knowledge  of  assessment  policies  and  practices  of  the 
organization,  including  restrictions  on  how  assessment 
data  are  to  be  used. 

6.  Knowledge  of  feedback  procedures  where  approp  late. 

In  addition,  some  measurement  is  needed  indicating  that  the 
individual  being  trained  has  the  capability  of  functioning  as  an 
assessor.  The  actual  measurement  of  assessor  performance  may  vary 
and  could,  for  example,  include  data  in  terms  of  rating  performance, 
critiques  of  assessor  reports,  observation  as  an  evaluator,  etc. 
What  is  important  is  that  assessor  performance  is  evaluated  to  in¬ 
sure  that  individuals  are  sufficiently  trained  to  function  as 
assessors,  prior  to  their  actual  duties,  and  that  such  performance 
is  periodically  monitored  to  insure  that  skills  learned  in  training 
are  applied. 


Informed  Consent  on  the  Part  of  Participants 

Informed  consent  is  a  fundamental  concern  in  conducting  an  assessment 
center  program.  This  means  that  the  participant  is  given  sufficient 
information  prior  to  assessment  to  evaluate  intelligently  the  nature 
of  the  program  and  the  consequences  of  attending  or  not  attending  a 
center.  While  organizations  have  the  right  to  require  participation 
in  an  assessment  program  as  a  condition  of  employment  or  advancement, 
individuals  should  not  simply  be  "sent"  to  a  center  with  little  aware¬ 
ness  of  why  they  are  going.  Rather,  they  should  be  provided  with 
sufficient  information  to  decide  whether  or  not  they  should  attend. 

While  the  actual  information  provided  will  vary  from  organization 
to  organization,  the  following  basic  information  should  be  given  to 
all  prospective  participants: 

1.  The  purpose  of  the  center  and  the  objectives  of  the  program. 

2.  How  individuals  are  selected  to  participate  in  the  center. 

3.  General  information  about  the  assessors  -  the  composition 
of  the  staff  and  their  training, 

4.  General  information  concerning  the  assessment  process 
itself.  This  should  include  a  description  of  the  techniques 
and  how  the  results  will  be  used,  the  kind  of  feedback  given, 

5.  Reassessment  policy. 

It  is  recognized  that  many  assessment  center  programs  have  descrip¬ 
tive  names  or  titles  which  are  often  neutral  or  purposefully  general. 
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This  is  an  acceptable  practice.  However,  it  would  he  inappropriate 
to  suggest  to  participants  that  the  assessment  center  is  for  personal 
development  or  training  when  the  clear  intent  is  for  selection  or 
management  staffing. 


6.  Use  of  Assessment  Center  Data 


One  characteristic  of  an  assessment  center  is  the  volume  of  data 
produced.  There  are  many  different  forms  of  assessment  data, 
ranging  for  example,  from  observer  notes,  to  reports  on  performance 
in  the  assessment  techniques,  to  assessor  ratings  and  reports  pre¬ 
pared  for  management.  The  preceding  is  not  exhaustive  and  could 
also  include  participant  and  peer  reports  and  observations,  biograph¬ 
ical  and  test  data,  etc. 

The  specific  purpose  of  the  reports  and  data  obtained  by  the 
assessment  center  should  be  clearly  established.  This  will  include 
a  statement  concerning  individuals  who  will  have  access  to  assess¬ 
ment  data,  the  kind  of  information  they  will  receive  and  the  format 
that  will  oe  provided. 

The  recipient  of  assessment  data  will  be  given  sufficient  inform¬ 
ation  or  training  so  that  the  data  provided  can  be  clearly  inter¬ 
preted.  This  will  include  an  estimate  of  the  relevance  of  current 
assessment  data  for  the  use  in  the  future. 

The  individual  assessed  should  be  informed  of  how  the  assessment 
data  are  to  be  used.  This  will  include: 

1.  Who  has  access  to  assessment  reports. 

2.  Whether  participants  will  normally  receive  feedback 
concerning  assessment  performance.  If  not,  provisions 
must  be  made  to  provide  such  information  upon  specific 
request. 

3.  How  long  assessment  information  will  be  retained  for 
operational  use  (as  opposed  to  research  use). 


7.  Validation  Issues 


A  major  factor  in  the  widespread  acceptance  and  use  of  assessment 
centers  is  directly  related  to  an  emphasis  on  sound  validation 
research.  Numerous  studies  have  been  conducted  and  reported  in  the 
professional  literature  demonstrating  the  validity  of  the  assessment 
center  process  in  a  variety  of  organizational  settings. 
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The  historical  record  of  the  validity  of  this  process,  however, 
cannot  be  taken  as  a  guarantee  that  a  given  assessment  program 
will  or  will  not  be  valid  in  a  given  setting.  Because  of  this, 
each  user  must  ascertain  the  validity  of  the  program  as  applied 
in  one's  organization.  The  technical  standards  and  principles 
for  validation  are  well  documented  and  appear  in  "Principles  for 
the  Validation  and  Use  of  Personnel  Selection  Procedures"  prepared 
by  the  Division  of  Industrial  and  Organizational  Psychology, 

American  Psychological  Association,  1975  and  "Standards  for  Educa¬ 
tional  and  Psychological  Tests  and  Manuals"  prepared  by  the  American 
Psychological  Association,  1974. 

In  addition  to  the  above  standards,  which  include  provisions 
related  to  demonstrating  fairness  and  validity;  some  specific 
guidelines  are  provided  for  assessment  center  programs.  These 
includes 

1.  The  ability  to  document  the  selection  of  dimensions, 
attributes  or  qualities  evaluated  in  the  center. 

2.  The  ability  to  document  thu  relationship  of  assessment 
center  techniques  to  specific  dimensions,  attributes 
or  qualities  evaluated. 

3.  The  ability  to  document  the  demographic  composition  of 
the  assessment  staff  as  representative  of  the  group  of 
individuals  assessed. 


8.  Concluding  Statement 

It  became  obvious  in  developing  these  standards  that  the  standards 
should  serve  as  guidelines  rather  than  doctrine.  Rather  than 
create  a  set  of  standards  that  become  ends  in  themselves,  the 
authors  attempted  to  provide  a  series  of  general  principles  which 
can  apply  to  both  managers  and  professionals  using  this  technique. 
Those  standards  should  enable  the  assessment  center  professionals 
to  create,  implement  and  maintain  assessment  center  programs  that 
protect  the  rights  of  individuals  while  meeting  organizational 
needs  at  the  same  time. 
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DEPARTMENT  OF  THE  AIR  FORCE 

HEAOOUARTtRS  AERON*  L'TICAL  SYSTEMS  DIVISION  I  AFSC  > 
WRIGHT  -FATT  £R50N  AIR  FORCE  BASE  OHIO  4S4S3 


vf  ■  V 


AT-TN  OF  ASD/DPCU 


*u»j«ct  Supervisory  Abilities  Inventory 


18  AUG  1971 


1.  The  Air  Force  Systems  Command  is  planning  to  test  a  new  method 
of  identifying  and  assessing  potential  candidates  for  first  level 
supervisory  positions.  Your  assistance  is  requested  in  identifying 
the  abilities  essential  to  effective  performance  as  a  first  level 
supervisor  in  your  organisation.  Please  complete  the  attached 
Supervisory  Abilities  Inventory  and  return  to  ASD/DPCU  by  30  August 


2.  In  determining  supervisor'/  effectiveness  there  are  many  personal 
characteristics  considered  to  be  important.  Generally  speaking, 
these  can  be  placed  in  one  of  the  following  areas: 

-  Supervisory  Traits 

-  Supervisory  Knowledge 

-  Supervisory  Abilities 

Supervisory  traits  are  characteristics  which  relate  to  such 
factors  as  loyalty,  integrity,  attitude,  honesty,  etc. 
Supervisory  knowledge  characteristics  are  such  factors  as  job 
knowledge,  and  work  experience.  Supervisory  abilities  are 
characteristics  which  enable  the  individual  to  effectively 
manage  his  work,  and  the  work  of  others. 

In  order  to  develop  a  Supervisory  Candidate  Assessment  Center 
for  use  in  the  Air  Force  Systems  Command,  we  are  interested 
in  your  views  concerning  abilities  and  skills  needed  to  become 
an  effective  supervisor. 

Attached  you  'will  find  a  li3t  of  supervisory  abilities.  Please 
rate  these  in  terms  of  the  importance  you  place  on  these  areas 
among  your  first  level  subordinates.  Net  all  of  these  ability 
areas  are  equally  important.  Sate  these  abilities  as  follows: 

Critically  Important:  5 

Somewhat  Important :  U 

Usually  Helpful:  3 

Somewhat  Unimport  ant :  2 

Net  Necessary:  1 


I 


SUPERVISORY  ABILITIES  INVENTORY 


Name i _ _  Grade: 

Title: 

Organisational  Symbol:  -'ate: 


DIRECTIONS:  Please  rate  each  of  the  following  abilities  in  terms  of  importance 
in  performing  as  a  first  level  supervisor  in  your  organization.  You  may  add 
any  other  ability  area  not  covered  in  the  list  which  you  feel  is  important. 
Please  describe  or  define  this  ability  and  then  rate  its  importance.  Rate  the 
abilities  using  the  numerical  3cale  shown: 

Critically  Important:  5 
Somewhat  Important:  4 

Usually  Helpful:  3 

Somewhat  Unimportant:  2 
Not  Necessary:  1 

(Note:  The  abilities  shown  are  not  equally  important  and  are  not  listed  in 
any  particular  order  of  importance.) 

RATING 


FLEXIBILITY :  Able  to  adjust  to  new  situations  easily 


ORGANIZING  SKILLS:  Can  plan  and  organize  work  of  self  and  others. 
ORAL  SKILLS:  Can  express  ideas  orally. 

WRITTEN  SKILLS:  Can  express  ideas  in.  writing. 

ENERGY:  maintains  a  high  level  of  activity. 

PR05LEM  SOLVING:  Abla  to  find  best  solution  easily. 

LEADERSHIP:  Able  to  get  others  to  work  effectively  in  a  group. 
PERSUAS IVENESS :  Able  to  sell  his  point  of  view  to  others 
DECISION  MAKING :  Makes  decisions  of  high  quality 
f QRCEFULNE5S :  Able  to  command  attention  from  others 


ORIGINALITv:  Able  to  approach  problems  in  a  new  way 

ACQUIRING  INFORMATION:  Able  to  obtain  information 

RISK  TAKING:  Willing  to  take  risk/responsibility  to  ar nieve 
objectives. 

STRESS  TOLERANCE:  Able  to  respond  appropriately  to  stressful 
conditions 
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AIR  FORCE  ASSESSMENT  CENTER 


Analysis  of  Supervisory  Dimensions 


J.  L.  Moses, 


Ph.D 


October  1973 


Introduction 


A  special  study  was  conducted  at  UJright-Patterson  Air  Force  Base 
in  order  to  determine  those  factors  viewed  by  management  personnel  as 
important  for  success  as  a  first  level  manager.  This  study  was  the 
first  step  in  designing  an  Assessment  Center  for  the  Air  Force  Systems 
Command.  In  addition  to  determining  supervisory  dimensions,  the  stuoy 
was  designed  to  analyize  whether  any  significant  differences  in  the 
characteristics  required  to  be  a  successful  supervisor  existed  among 
personnel  in  two  organizations  planning  to  use  an  Assessment  Center,  ASD 
and  AFAL. 

A  special  rating  form,  The  Supervisory  Abilities  Inventory,  was 
developed.  This  consisted  of  a  series  of  14  different  abilities  generally 
found  related  to  supervisory  success.  These  abilities  and  their  definitions 
are: 

-  Flexibility:  Able  to  adjust  to  new  situations  easily 

-  Organizing  Skills:  Can  plan  and  organize  work  of  self  and  others. 

-  Oral  Skills:  Can  clearly  express  ideas  orally. 

-  Written  Skills:  Can  clearly  express  ideas  in  writing. 

-  Energy:  maintains  a  high  level  of  activity. 

-  Problem  Solving:  Able  to  find  best  solution  easily. 

-  Leadership:  Able  to  get  others  to  work  effectively  in  a  group. 

-  Persuasiveness:  Able  to  sell  his  point  of  view  to  others. 

-  Oecision  Making:  Making  decisions  of  high  quality. 

-  Forcefulness:  Able  to  command  attention  from  others. 


158 


-  Originality:  Able  to  approach  problems  in  a  new  way, 

-  Acquiring  Information:  Able  to  obtain  information. 

-  Risk  Taking:  Willing  to  take  risks/responsibility  to  achieve 

objectives . 

-  Stress  Tolerance:  Able  to  respond  appropriately  to  stressful 

conditions. 

The  Supervisory  Abilities  Inventory  also  had  space  provided  for 
the  supervisor  to  offer  additional  abilities.  The  Inventory  was  admin¬ 
istered  to  supervisory  personnel  in  the  ASD  and  AFAL  organizations  at 
Ulright-Patterson  Air  Force  Base.  Each  individual  was  given  a  cover  letter 
explaining  the  purpose  of  the  inventory,  and  was  asked  to  rate  each  of  the 
fourteen  dimensions  on  a  five  point  scale  ranging  from  5:  critically 
important;  4:  somewhat  important;  3:  usually  helpful;  2:  somewhat  unim¬ 
portant;  1:  not  necessary.  The  supervisor  was  also  advised  that  he  could 
add  any  other  ability  areas  nut  covered  in  the  Inventory  which  he  felt  was 
important.  As  noted,  space  was  provided  for  this  on  the  Inventory, 
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Supervisors  Studied 


A  total  of  147  supervisors  participated  in  this  study.  Of  these  , 
140  mere  GS  level  12  or  higher.  A  total  of  60  supervisors  from  ASO  and 
87  from  AFAL  completed  the  Supervisory  Abilities  Inventory.  This  was  a 
large  sample  and  uias  representative  of  supervisory  personnel  in  each 
organization.  A  breakdown  of  the  participants  by  GS  level  and  organization 
is  presented  in  Table  1. 


TA8LE  1 

NUMBER  OF  PARTICIPANTS  BY  ORGANIZATION  AND  LEVEL 


GS  LEVEL 

ASP 

AFAL 

16 

- 

1 

15 

15 

11 

14 

35 

35 

13 

10 

22 

12 

- 

11 

11  or  less 

- 

7 

TOTAL 

60 

87 

140 


Analysis  of  Results 

All  of  the  14  dimensions  listed  on  the  Supervisory  Abilities 
Inventory  mere  judged  to  be  at  least  •■usually  helpful”  for  the  supervisory 
success  as  a  first  level  manager,  the  follo.ing  abilities,  listed  in  their 

order  of  importance  were  seen  as: 

-  Critically  important:  Leadership 

Organization  skills 
Decision  making 

-  Somewhat  important:  Oral  skills 

Written  skills 

F lexibility 

Stress  tolerance 

Persuasiveness 

Risk  Taking 

Problem  solving 

Acquiring  information 


-  Usually  helpful: 


F  orcef  ulness 
Energy 


The  mean  ratings  for  the  AFAU  and  ABC  supervisors  are  presented 


in  Table  2. 


TABLE  2 


(YltEAN  RATINGS  GF  AFAL  AND  ASP 


SUPERVISORS  OF 

RATINGS  GN  SUPERVISORY 

ABILITIES  INVENTORY 

Dimension 

AFAL 

(N=8?) 

ASD 

(N=60) 

Leadership 

4.80 

4.86 

Organizing  Skills 

4.68 

4.03 

Oecision  (flaking 

4  .  A  5 

4.55 

Oral  Skills 

4.44 

4.26 

Written  Skills 

4.33 

4.18 

F lexibility 

4.21 

4.25 

Stress  Tolerance 

4.20 

4.23 

Persuasiveness 

4.14 

4.08 

Risk  Taking 

4.09 

4.05 

Problem  Solving 

CJ 

• 

3.93 

Acquiring  Information 

3.80 

3.81 

Qriginality 

3.79 

3.50 

F  orcef ulness 

3.57 

3.71 

Energy 

3.56 

3.41 

As  seen  in  Table  2,  there  is  a  high  degree  of  agreement  between 
the  ratings  made  by  ASD  and  AFAL  supervisors.  The  mean  ratings  were 
converted  to  rnak  order  scores  and  are  presented  in  Table  3. 


TABLE  3 


RANK  ORDER  OF  SUPERVISORY  ABILITIES  BY 
ASO  AND  AFAL  SUPERVISORS 


Ranking 


dimension 

AFAL 

ASP 

Leadership 

1 

1 

Organizing  Skills 

2 

2 

Decision  iilaking 

3 

3 

Oral  Skills 

4 

4 

Written  Skills 

5 

7 

F lexibility 

6 

5 

Stress  Tolerance 

7 

6 

Persuasiveness 

a 

8 

Risk  Taking 

9 

9 

Problem  Solving 

10 

10 

Acquiring  Information 

11 

11 

Originality 

12 

13 

F  orcef ulness 

13 

12 

Energy 

14 

14 

The  ranking  indicates  virtually  complete  agreement  of  the  relative 
importance  of  these  attributes  by  supervisors  in  both  organizations.  A 
rank  order  correlation  coefficient  was  computed,  and  a  correlation  of 
.993  uias  obtained  between  the  rankings  shown  in  Table  3.  For  all  practical 
purposes,  supervisors  in  both  these  organizations  hold  views  concerning  the 
attributes  required  to  be  a  successful  first  level  manager. 
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The  data  was  examined  in  greater  detail  to  determine  whether 
different  rating  patterns  existed  as  a  function  of  management  level.  These 
are  presented  in  Table  4.  The  ratings  of  the  one  GS-16  Supervisor  in  AfAL 
were  combined  with  the  6S-15  group.  The  ratings  of  the  supervisors  at 
GS-11  or  1  ss  were  not  included  as  the  number  of  cases  (7)  was  too  small 
to  constitute  a  meaningful  subgroup. 

As  can  be  seen,  there  is  generally  close  agreement  for  supervisors 
at  different  levels  in  both  organizations.  With  a  f ew  exceptions,  first 
level  incumbents,  their  supervisors  and  their  supervisors’  bosses  tend  to 
substantially  agree  on  the  abilities  needed  for  success  as  a  first  level 
supervisor.  To  some  axtpnt,  there  is  more  disagreement  among  supervisors 
in  the  same  organization  concerning  the  relative  rating  given  a  dimension, 
than  supervisors  in  AfAL  when  compared  to  their  counterpart  in  A3D.  Table 
5  shows  these  comparisons  more  clearly.  It  presents  the  differnces  between 
mean  ratings  for  the  A5Q  and  AFAL  groups,  as  well  as  differences  within 
ASO  and  AFAL  between  higher  and  lower  GS  level  supervisors.  For  AFAL, 
differences  are  shown  between  supervisors  at  GS  level  15  and  12;  for  A5D 
between  GS  levels  15  and  13. 


TABLE  4 


WEAN  RATINGS  BY  G.5.  LEVEL  OF  AFAL  AND  A5D  SUPERVISORS 

AFAL  ASD 


GS 

GS 

GS 

GS 

GS 

GS 

GS 

15 

14 

13 

12 

15 

14 

13 

Leadership 

4.66 

4.94 

4.50 

5.00 

4.93 

4.85 

■P* 

o 

CD 

CD 

organizing  Skills 

4.66 

4.73 

4.54 

4.72 

4.87 

4.51 

4.70 

Oecision  Waking 

4.41 

4.42 

4.40 

4.59 

4.46 

4,54 

4.70 

Oral  Skills 

4.58 

4.45 

4.54 

4.36 

4.26 

4.28 

4.30 

Written  Skills 

4.33 

4.51 

4.31 

4.09 

4.06 

4.31 

3.90 

Flexibility 

4.33 

4.20 

4.00 

4.36 

4.13 

4.37 

4.00 

Stress  Tolerance 

4.16 

4.28 

4.00 

4.00 

4.20 

4.2E 

4.10 

Persuasiveness 

4.58 

3.97 

4.04 

4.18 

4.00 

4.20 

3.80 

Risk  Taking 

3.83 

3.97 

3.90 

4.63 

4.06 

4.11 

3. 80 

Problem  Solving 

4.G8 

3.82 

4.13 

4.09 

4.00 

3.65 

4.10 

Acquiring 

In  formation 

3.75 

3.80 

3.59 

3.72 

3.93 

3.80 

3.70 

Originality 

3.58 

3.54 

4.09 

3.81 

3.26 

3.60 

3.50 

Forcefulness 

3.66 

3.51 

3.40 

3.81 

3.60 

3.82 

3.20 

Energy 

3.66 

3.31 

3.66 

3.72 

3.40 

3.48 

3.20 

TABLE  5 


DIFFERENCES 

8E  TUIEEN  MEAN 

RATINGS  BY  ASO  AND  AFAL 

SUPERVISORS  AS 

A  FUNCTION  OF 

ORGANIZATION  AND  G5 

LEVEL 

Dimension 

AFAL 

vs 

ASD 

AFAL 

GS-15 

vs 

GS-12 

ASO 

GS-15 

vs 

GS-13 

Leadership 

.06 

.34 

.13 

Organizing  Skills 

.05 

.06 

.1? 

Decision  (flaking 

.10 

.18 

.24 

Oral  Skills 

.16 

.22 

.04 

Written  Skills 

.15 

.24 

.16 

F lexihility 

.04 

.03 

.13 

Stress  Tolerance 

.03 

.15 

.10 

Persuasiveness 

.06 

.40 

.20 

Risk  Taking 

.04 

1.20 

.26 

Problem  Solving 

.11 

.01 

.10 

Acquiring  Information 

.01 

.03 

.23 

Originality 

.29 

.23 

.24 

F  orcefulnass 

.14 

.15 

.60 

Energy 

.15 

.06 

.20 
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As  noted,  the  differences  in  the  mean  rating  between  AFAL  and  ASO 
supervisors  are  slight.  For  example,  the  mean  rating  difference  for  the 
leadership  dimension  was  six  one/hundredth's.  These  differences  ranged 
from  .01  to  ,29  of  a  rating  scale  unit.  Not  only  do  supervisors  in  both 
of  these  organizations  rank  these  abilities  similarly,  they  also  rate 
them  substantially  the  same. 

For  the  most  part,  lower  and  higher  level  supervisors  agreed  on 
the  importance  of  these  dimensions,  regardless  of  whether  these  supervisors 
were  in  ASO  or  AFAL.  There  are  only  two  abilities  viewed  considerably 
different  by  supervisors  in  each  organization.  AFAL  higher  level  super¬ 
visors  rated  risk  taking  less  important  than  did  lower  level  supervisors. 

ASO  higher  level  supervisors  saw  forcefulness  as  more  important  than  did 
lower  level  supervisors.  With  these  two  conceptions,  all  of  the  remaining 
ratings  substantially  agreed. 

Analysis  of  Additional  Comments  made  by  supervisors 

As  noted  earlier,  space  was  provided  on  the  Supervisory  Abilities 
Inventory  for  the  supervisors  to  add  additional  dimensions  not  considered 
in  the  Inventory.  A  a  general  rule,  most  supervisors  did  not  add  any 
dimensions,  evidently  feeling  that  the  list  provided  in  the  Supervisors 
Abilities  Inventory  was  sufficient.  A  total  of  eighteen  supervisors  in 
ASD  and  twelve  supervisors  in  AFAL  provided  additional  dimensions.  These 
included  such  diverse  areas  as  "accept  direction”  to  "productivity".  One 
area  was  mentioned  by  several  supervisors  and  might  be  included  if  techniques 
can  be  developed  to  reliably  measure  this  attribute.  This  area,  empathy, 
was  defined  as  a  concern  for  those  supervised. 
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Conclusion 


The  resuius  of  this  analysis  indicate  that  supervisors  in  both  ASD 
anL  vie..  ' hectically  the  qualities  needed  for  success  as  a  first  leve 

superv'sor  i1-  their  respective  organizations.  These  abilities  are: 
leaders!,  p,  o.danizing  skills,  decision  making,  oral  skills,  written  skill 
flexibility,  stress  tolerance,  persuasiveness,  risk  taking,  problem 
solving,  acquiring  information,  originality,  forcefulness,  ana  energy. 

An  additional  quality,  empathy,  was  added  by  some  supervisors. 

No  significant  differences  were  noted  in  ratings  or  rankings  of 
ratings  made  by  supervisors  in  ASD  and  AFAI. .  This  means  thac  a  similar 
assessment  program  available  to  bath  organizations  can  and  should  be 
utilized.  It  is  also  passible  to  assess  individuals  simultaneously  from 
both  organizations  in  a  single  assessment  center  if  c  l  .sired. 

The  Supervisory  Abilities  Inventory,  having  proved  useful  in  deter 
mining  the  relative  rank  order  of  importance  perceived  by  incumbent  super¬ 
visors,  might  make  for  a  useful  assessment  technique.  It  can  have 
particular  merit  in  counseling  individuals  concerning  the  relative  sig¬ 
nificance  of  supervisory  abilities  needed  at  both  ASO  and  AFAL. 


APPENDIX  D 
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ASSESSMENT  CENTER  TRIAL  RUN 


Quest ionna I  re 

A.  To  ref resh  your  memory,  the  activities  in  the  2-day  Assessment  Center 
are  briefly  described  as  follows: 

1.  Participate  as  task  force  members  to  make  recommendations  for 
ranking  supervisory  attributes. 

2.  Group  exercise  where  each  member  must  persuade  his  peers  to  adopt 
city  budget  recommendation  assigned  at  random  at  beginning  of  exercise. 

3.  Participate  on  Committee  which  must  resolve  four  varied  management 
problems . 

4.  Individual  fact-finding  exercise  to  obtain  relevant  information 
needed  to  make  decision. 

5.  Prepare  formal  written  proposal. 

6.  Personal  interview  with  Assessor. 

7.  Complete  biographical  questionnaire. 

8.  Complete  Career  Development  Inventory. 

9.  Work  preference  tests  (three). 

10.  Feedback  Interview  with  Assessor  a  few  days  after  Assessment  Center 
exercises  were  completed. 

B.  A  series  of  questions  will  now  be  addressed  to  each  of  the  above  exercises. 
Please  answer  to  the  best  of  your  ability  by  placing  a  circle  around  appropri¬ 
ate  answer,  or  by  providing  comment. 

1.  Task  Force  Member  Exercise 


a.  Were  the  Instructions  clear  and  problem  structured  so  that  you 
could  approach  problem  In  rational  manner? 

9U_  YES  2%  NO  6%  DON'T  KNOW 

b.  Was  problem  realistic  enough  to  provide  discussion  interplay 
between  the  group  members? 

Sb%  YES  1%  NO  4*  DON'T  KNOW 
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c.  Was  adequate  time  allowed  for  this  problem  during  the 
Assessment  Center7 

83%  YES  13%  NO  4%  DON'T  KNOW 

d.  Did  the  ranking  of  yourself  and  other  members  of  the  group  at 
the  end  of  the  session  provide  Insight  into  the  exercise? 

57%  YES  21%  NO  21%  DON'T  KNOW 

e.  Old  the  presence  of  the  Assessor  In  the  room  bother  you  after 
the  first  few  minutes? 

6%  YES  89%  NO  4%  DON'T  KNOW 

f.  Was  the  problem  too  difficult? 

2%  YES  96%  NO  2%  DON'T  KNOW 


City  Budget  Recommendation 

a.  Were  the  instructions  clear  and  problem  structured  so  that 
you  could  approach  problem  in  rational  manner? 

91%  YES  9%  NO  DON'T  KNOW 

b.  Was  problem  realistic  enough  to  provide  discussion  interplay 
between  the  group  members? 

87%  YES  13%  NO  DON'T  KNOW 

c.  Was  adequate  time  allowed  for  this  problem  during  the 
Assessment  Center? 

66%  YES  34%  NO  DON'T  KNOW 

d.  Did  the  ranking  of  yourself  and  other  members  of  the  group 
at  the  end  of  the  session  provide  Insight  Into  the  exercise? 

55%  YES  28%  NO  17%  DON'T  KNOW 

e.  Did  the  presence  of  the  Assesses  In  the  room  bother  vou  after 
the  first  few  minutes? 

4%  YES  94%  NO  DON'T  KNOW 
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f.  Was  the  problem  too  difficult? 


2*  YES  91  *  NO  oZ  DON'T  KNOW 


3.  Committee  to  Solve  Four  Management  Problems 

a.  Were  the  Instructions  clear  and  problem  structured  so  that  you 
could  approach  problem  In  a  rational  manner? 

100*  YES  NO  DON'T  KNOW 

b.  Was  problem  realistic  enough  to  provide  discussion  Interplay 
between  the  group  members? 

98*  YES  2*  NO  DON'T  KNOW 

c.  Was  adequate  time  allowed  for  this  problem  during  the 
Assessment  Center? 

8l*  YES  15*  NO  A*  DON'T  KNOW 

d.  Did  the  ranking  of  yourself  and  other  members  of  the  group  at 
the  end  of  the  session  provide  Insight  into  the  exercise? 

57*  YES  26*  NO  17*  DON'T  KNOW 


e.  Did  the  presence  of  the  Assessors  In  the  room  bother  you  after 
the  first  few  minutes? 

6*  YES  94*  NO  DON’T  KNOW 

f.  Was  the  problem  too  difficult? 

YES  100*  NO  DON'T  KNOW 

4.  Fact-finding  Exercise  -  Continuation  of  Research  Project 

a.  Were  the  instructions  clear  and  problem  structured  so  that  you 
could  approach  problem  In  rational  manner? 

79*  YES  15*  NO  6*  DON'T  KNOW 

b.  Did  the  Assessor  respond  to  questions  promptly  and  provide 
additional  Information  In  a  complete  manner? 

77*  YES  19*  NO  4*  DON'T  KNOW 


1b2 
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c.  Old  you  feel  frustrated,  angered,  or  pressured  during  the 
10-mlnute  questioning  period? 


14* 

frustrated 

*  ( 

4* 

angered 

1  (%4 

1  i 

f'; 

t 

20* 

pressured 

61* 

none  of  the  above 

-  i 

1 

i' 

it  on 

Changing  Organization 

il' 

*  »  t 

a.  Was  the  problem  thought-provoking  to  the  pc-. .it  the:  you  were 
able  to  present  your  Ideas  in  writing  without  undue  difficulty? 

79*  YES  21*  NO  DON'T  KNOW 

b.  Was  the  writing  assignment  too  easy? 

2*  YES  89*  NO  9*  DON'T  KNOW 


6.  Interview  With  Assessor  During  Assessment  Center 

a.  Was  adequate  time  allowed  for  the  interview? 

91*  YES  4*  NO  4*  DON'T  KNOW 

b.  Was  a  meaningful  discussion  held  between  you  and  the  Assessor 
during  the  interview? 

81*  YES  11*  NO  9*  DON'T  KNOW 

c.  The  Intel  views  are  structured  to  delve  into  a  number  of  areas 
which  are  normally  considered  persons!.  Was  the  interview  too 
persona  1 ? 

YES  98*  NO  2*  DON’T  KNOW 

d.  The  Interview  Is  fairly  comprehensive.  Have  you  received 

a  more  In-depth  Interview  with  any  of  the  other  merit  promotion 
or  performance  evaluation  programs? 

9*  YES  85*  NO  6*  DON'T  KNOW 
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7 .  The  Completion  of  the  Biographical  Questionnaire 

a.  The  questions  on  the  "Background  Questionnaire"  are  fairly 
comprehensive  considering  your  academic  and  personal  life.  Are 
they  too  personal? 

YES  96%  NO  4%  DON'T  KNOW 


b.  Was  the  information  assembled  sufficient  to  provide  a  good 
discussion  during  the  interview  with  the  Assessor? 

74%  yes  15%  NO  111  DON'T  KNOW 


c.  Would  you  recommend  that  the  "Questionnaire"  be  expanded  to 
include  more  background  information  on  your  academic  and  work 
experience? 

38%  YES  401  NO  211  DON'T  KNOW 


8.  Career  Development  Inventory 

a.  Did  you  bring  the  Career  Development  Inventory  with  you  when 
you  met  with  the  Assessor  for  the  feedback  interview? 

68%  YES  19%  NO  13%  DON'T  KNOW 

b.  Were  the  questions  realistic  In  assembling  data  on  past 
accomplishments  and  future  plans? 

57%  YES  17%  NO  261  DON'T  KNOW 

c.  Have  you  completed  any  action  since  the  Assessment  Center  in 
the  Career  Development  area? 

43%  YES  51%  NO  6%  DON'T  KNOW 


d.  If  yes,  was  the  action  prompted  by  something  that  happened 
during  the  Assessment  Center? 

38%  YES  46%  NO  17%  DON'T  KNOW 

9.  Work  Preference  Tests  (three) 

The  work  preference  tests  are  used  to  determine  preference  in 
supervisory  style.  The  results  of  the  tests  will  be  used  at  a  later 
date  afrcr  a  larger  group  of  Assessees  have  been  tested. 
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Feedback  Interview  with  Assessor 


a.  The  Assessors  were  to  contact  you  3  to  5  days  after  the 
Assessment  Center  was  completed  for  the  Interview.  TOY  and  other 
work  commitments  caused  delay  in  this  proposed  time  frame  for 
some  Assessees.  Please  indicate  the  approximate  time  delay 
before  your  Interview. 


37% 

3  - 

5  days 

43% 

6  - 

12  days 

20% 

More 

than  12  days 

b.  Did  the  Assessor  give  you  a  report  !n  writing  as  to  the 
results  of  the  Assessment  Center? 

13%  YES  87%  NO  DON'T  KNOW 

c.  Did  the  Assessor  present  the  information  in  a  manner  that 
Indicated  he  had  preplanned  for  the  interview? 

77%  YES  17%  NO  6%  DON'T  KNOW 

d.  What  was  the  approximated  time  used  for  the  feedback  Interview? 

30%  20  minutes  or  less 

55%  20-40  minutes 

13%  1*0-60  minutes 

60-90  minutes 
1%  90  minutes  or  more 

e.  The  ratings  are  labeled  "More  than  Acceptable,"  "Acceptable," 

"Questionable,"  and  "Not  Acceptable."  Did  the  Assessor  Indicate 
what  rating  you  had  received  as  a  result  of  the  Assessment  Center? 

70%  YES  28%  NO  2%  DON'T  KNOW 

f.  As  you  know,  Assessees  receive  feedback  information  during  the 
Asse*<m«nt  Center  by  observation  and  discussion  with  other  Assessees. 
Did  the  feedback  interview  with  the  Assessor  correlate  with  what 
you  expected  the  Assessment  Center  to  predict? 


66%  YES 


15% 


NO 


1 9%  DON ' T  KNOW 
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g.  Old  the  Assessor  respond  to  your  questions  openly  and  In 
knowledgeable  fashion? 

89%  YES  3%  NO  2%  OON'T  KNOW 


h.  Old  you  have  confidence  at  the  time  of  the  interview  in  the 
information  presented  by  the  Assessor? 

78%  YES  16%  NO  1%  DON'T  KNOW 


C.  The  next  portion  of  the  survey  concerns  the  proposed  distribution  and  use 
of  the  Assessment  Center  results.  Approximately  300  employees  will  be 
assessed  during  the  coming  year. 

1.  The  Assessment  Center  will  be  used  as  an  Indicator  for  all  first 
level  supervisory  positions  at  the  GS-13  level  and  higher  in  the  Merit 
Promotion  Program.  Attendance  will  not  be  mandatory  and  supervisors 
may  select  a  person  regardless  of  his  Assessment  Center  rating.  The 
supervisor  will  be  asked  to  document  hts  rationale  for  selecting  a 
person  who  refuses  to  attend  the  Assessment  Center,  or  why  he  might 
choose  an  employee  with  a  "Not  Acceptable"  or  "Questionable"  rating 
over  those  who  are  available  and  have  a  "More  than  Acceptable"  or 
"Acceptable"  rating,  Oo  you  concur  In  general  with  the  above  plan? 

81%  YES  13%  NO  6%  DON'T  KNOW 

2.  Each  Assessee  will  have  the  option  of  Inviting  or  not  inviting 
his  first  level  supervisor  to  sit  In  on  the  feedback  Interview  with 
the  Assessor.  Oo  you  agree? 

7*4%  YES  17%  NO  9%  DON'T  KNOW 


3.  The  "feedback  report"  prepared  by  the  Assessor,  which  consolidates 
all  the  fine  ngs  developed  during  the  Assessment  tenter,  will  be  filed 
In  the  Official  Personnel  Folder  (OPF).  It  will  be  filed  in  a  "secure" 
folder  with  other  test  material,  vouchers  from  past  employers  and 
health  records.  This  folder  Is  routinely  pulled  from  Official  Per¬ 
sonnel  Folder  when  you  review  your  own.  One  copy  will  also  be  provided 
to  the  first  level  supervisor  Immediately  after  the  feedback  interview. 
Do  you  agree  or  have  comments  concerning  the  above  distribution  of 
feedback  reports? 

56%  YES  31%  NO  13%  DON'T  KNOW 
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4.  Assessees  will  be  allowed  to  retake  the  Assessment  Center  \ 

(after  6  months)  to  improve  the  original  rating.  The  rating  from 

the  last  Assessment  Center  attended  will  be  the  rating  for  record 
and  will  oe  used  whether  It  Is  higher  or  lower.  Do  you  concur  In  this 
proposal ? 

38%  YES  55%  NO  6%  DON'T  KNOW 

5.  The  Assessment  Center  will  also  be  used  to  evaluate  "high 
potential"  employees  Identified  for  long-term  full-time  training 
and  other  high  expense  Career  Development  programs.  Do  you  agree? 

74%  YES  15%  NO  11%  DON'T  KNOW 


6.  "At  least  half  the  'worth'  of  the  Assessment  Center  Is  the 
benefit  the  Assessee  receives  personally  from  attending  the  Assessment 
Center."  Do  you  agree  with  this  statement? 

70%  YES  23%  NO  6%  DON'T  KNOW 


ASSESSOR  FEEDBACK  INTERVIEW  OUTLINE 


1.  Establish  rapport  -  Casual  conversation  to  establish  a  relaxed, 

informal  climate  and  reduce  passible  anxiety. 

2.  Ask  the  assessee  to  relate  thoughts  about  the  assessment  center 

and  its  value. 

3.  Explain  the  purpose  of  the  interview. 

a.  Help  the  individual  benefit  from  the  assessment  center  experience. 

b.  Aid  the  individual  in  forming  a  developmental  plan. 

4.  Review  and  discuss  the  assessment  process. 

a.  Assessors  observed  and  recorded  behavior. 

b.  Assessors  met  and  shared  observations  from  the  various 
exercises  and  discussed  each  individual  for  several  hours. 

c.  They  then  came  to  an  overall  judgment  of  strengths  and 
development  needs. 

d.  Comparisons  in  the  discussion  were  not  between  people 

in  the  assessment  center  but  against  established  benchmarks. 

5.  Provide  feedback  of  assessment  results  for  each  dimension/exererse . 

a.  Name  dimension  and  ask  assessee  to  explain  it. 

b.  State  the  official  definition  and  reach  consensus  with  the 
assessee. 

c.  Identify  and  briefly  describe  the  exercises  measuring  the 
dimension. 

d.  Ask  the  assessee  to  evaluate  own  performance  on  the  dimension. 

e.  Provide  the  assessor's  findings  concerning  the  assessee' s 
performance.  Support  with  examples  of  behavior  from  appropriate 
exercises. 

f.  Compare  and  contrast  performance  in  various  exercises  measuring 
the  dimensions. 

g.  Encourage  two-way  discussion  of  feedback. 
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h.  Ask  assessee  to  suggest  ways  to  improve  performance  of  the 
dimension,  if  any. 

i.  Present  recommendations  for  corrective  actions  or  remedial 
plans. 

j.  After  reviewing  all  the  dimensions,  give  summary  of  overall 
performance  in  each  exercise. 

k.  Ask  for  comments-.  Determine  if  the  assessee  understands  the 
information  provided  about  each  dimension/exercise. 

6.  Discuss  major  strengths. 

a.  Do  not  minimize  this  area. 

b.  Even  if  the  area  is  a  strength  it  may  oe  a  development 
opportunity  (A  strength  can  be  used  to  overcome  or  help 
develop  a  weak  area). 

7.  Discuss  major  developmental  needs. 

a.  Select  the  three  or  four  most  important  developmental  needs 
unless  several  developmental  needs  are  relatec  tc  each  other. 

b.  Choose  developmental  needs  which  can  be  developed. 

6.  Obtain  general  feedback  from  the  assessee  in  the  form  of  questions 
and  the  assessee's  reactions  to  the  assessor’s  observations  and 
recommendations. 

9.  Administer  the  post-assessment  center  questionnaire.  Close  the 
interview  on  a  high  note  by  emphasizing  strengths  and  the 
possibility  to  overcome  weaknesses. 


- 
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Captain  David  lift.  Komar  enlisted  in  the  Air  Force  on  25  Nov¬ 
ember,  1970  after  graduating  from  John  Carroll  University,  Cleveland, 
Ohio,  with  a  bachelor's  degree  in  physics.  He  served  as  a  Special 
Electronics  Technician  until  commissioned  in  1974  through  the  Officer 
Training  School  program.  As  an  officer  he  completed  Electronic 
Systems  Officer  training  and  then  served  as  a  maintenance  Supervisor 
with  SAC,  and  QIC,  Quality  Control  and  Chief,  Ground  Communications 
Operations  with  ACCOM.  Upon  completion  of  his  AF1T  assignment. 

Captain  Komar  was  assigned  to  the  Directorate  of  Requirements, 
Communication  Plectranics  Systems,  Headquarters  Tactical  Air  Command. 

Captain  William  M.  Wise  graduated  from  the  University  of 
Florida  with  a  bachelor's  degree  in  industrial  engineering  in  June  1974 
and  was  commissioned  through  the  Air  Force  ROTC  program.  His  initial 
assignment  was  to  the  Electronic  Systems  Division,  hanscom  AFB, 
Massachusetts,  where  he  was  a  project  engineer  on  the  Advanced  Air¬ 
borne  Command  Post  Program.  In  July  197?  he  was  assigned  to  the 
Aeronautical  Systems  Oivision,  UJright-Patterson  AFE,  where  he  was  a 
project  manager  on  the  Pave  Low  III  Night/Adverse  Weather  Rescue  Heli¬ 
copter  Program.  Captain  Wise  is  currently  assigned  to  the  Air  Force 
Technical  Applications  Center,  Patrick  AF5,  Florida. 


